2024-04-22T06:56:34.680 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-22T06:56:34.681 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/fs/snaps}} --archive /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159 --owner scheduled_leonidus@teuthology -- /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159/orig.config.yaml 2024-04-22T06:56:34.732 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T06:56:34.836 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159 branch: wip-lusov-quiescer description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/fs/snaps}} email: null first_in_suite: false job_id: '7668159' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 4362 sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 sleep_before_teardown: 0 subset: 222/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJVl31wtivxBEbjpTOCjOi5KvLj1vB5LcpWY56N+WMhEvOx32lnLtLoh1Bq8+LgeR9qUCXErSnly6rZdEGibL6Y= smithi145.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDTO262E0ybwmBW+GUof6kOF2K0vgwR1F/9MxdtBjNRvP52H+6lALEgMCoJarHGCC43MXkctFfZm4ttjS+xK7XQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - quiescer: null - workunit: clients: all: - fs/snaps timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-22_05:52:53 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-22T06:56:34.836 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa; will attempt to use it 2024-04-22T06:56:34.837 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks 2024-04-22T06:56:34.837 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-22T06:56:34.838 INFO:teuthology.task.internal:Checking packages... 2024-04-22T06:56:34.862 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '86f7587a5a09af35f0895e1a2d08527638fae697' 2024-04-22T06:56:34.862 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T06:56:34.862 INFO:teuthology.packaging:ref: None 2024-04-22T06:56:34.862 INFO:teuthology.packaging:tag: None 2024-04-22T06:56:34.862 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-22T06:56:34.862 INFO:teuthology.packaging:sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T06:56:34.862 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-22T06:56:35.031 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.g86f7587a 2024-04-22T06:56:35.033 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-22T06:56:35.038 INFO:teuthology.task.internal:no buildpackages task found 2024-04-22T06:56:35.038 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-22T06:56:35.044 INFO:teuthology.task.internal:Saving configuration 2024-04-22T06:56:35.062 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-22T06:56:35.067 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T06:56:35.094 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:47.198794', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJVl31wtivxBEbjpTOCjOi5KvLj1vB5LcpWY56N+WMhEvOx32lnLtLoh1Bq8+LgeR9qUCXErSnly6rZdEGibL6Y='} 2024-04-22T06:56:35.125 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:47.197371', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDTO262E0ybwmBW+GUof6kOF2K0vgwR1F/9MxdtBjNRvP52H+6lALEgMCoJarHGCC43MXkctFfZm4ttjS+xK7XQ='} 2024-04-22T06:56:35.125 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-22T06:56:35.132 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-22T06:56:35.132 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-22T06:56:35.132 INFO:teuthology.run_tasks:Running task console_log... 2024-04-22T06:56:35.219 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f04f26940d0>, signals=[15]) 2024-04-22T06:56:35.220 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-22T06:56:35.225 INFO:teuthology.task.internal:Opening connections... 2024-04-22T06:56:35.225 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-04-22T06:56:35.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T06:56:35.319 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2024-04-22T06:56:35.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T06:56:35.403 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-22T06:56:35.409 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-04-22T06:56:35.427 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-04-22T06:56:35.427 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-04-22T06:56:35.484 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-04-22T06:56:35.484 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-04-22T06:56:35.484 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-04-22T06:56:35.484 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-04-22T06:56:35.484 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-04-22T06:56:35.484 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-04-22T06:56:35.484 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T06:56:35.484 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-04-22T06:56:35.484 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-04-22T06:56:35.484 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T06:56:35.484 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-04-22T06:56:35.485 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T06:56:35.485 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T06:56:35.485 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T06:56:35.485 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-04-22T06:56:35.515 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2024-04-22T06:56:35.533 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2024-04-22T06:56:35.533 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2024-04-22T06:56:35.590 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2024-04-22T06:56:35.590 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="9" 2024-04-22T06:56:35.591 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2024-04-22T06:56:35.591 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2024-04-22T06:56:35.591 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="9" 2024-04-22T06:56:35.591 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el9" 2024-04-22T06:56:35.591 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T06:56:35.591 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2024-04-22T06:56:35.591 INFO:teuthology.orchestra.run.smithi145.stdout:LOGO="fedora-logo-icon" 2024-04-22T06:56:35.591 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T06:56:35.591 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2024-04-22T06:56:35.591 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T06:56:35.591 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T06:56:35.591 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T06:56:35.592 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2024-04-22T06:56:35.623 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-22T06:56:35.632 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-22T06:56:35.638 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-22T06:56:35.638 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-04-22T06:56:35.640 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2024-04-22T06:56:35.657 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-22T06:56:35.662 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-22T06:56:35.662 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-04-22T06:56:35.697 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2024-04-22T06:56:35.783 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-22T06:56:35.852 INFO:teuthology.run_tasks:Running task kernel... 2024-04-22T06:56:35.867 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-22T06:56:35.867 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-22T06:56:35.867 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T06:56:35.868 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T06:56:35.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:56:35.884 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-04-22T06:56:35.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:56:35.885 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-04-22T06:56:35.941 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-437.el9.x86_64 2024-04-22T06:56:35.941 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-437.el9.x86_64 2024-04-22T06:56:35.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2024-04-22T06:56:35.942 INFO:teuthology.orchestra.run.smithi145.stdout:5.14.0-437.el9.x86_64 2024-04-22T06:56:35.942 INFO:teuthology.task.kernel:Running kernel on smithi145: 5.14.0-437.el9.x86_64 2024-04-22T06:56:35.942 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum install -y kernel 2024-04-22T06:56:38.234 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-22T06:56:40.368 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-22T06:56:42.772 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 1.5 MB/s | 8.0 MB 00:05 2024-04-22T06:56:44.485 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-22T06:56:44.631 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-22T06:56:46.052 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 16 kB 00:00 2024-04-22T06:56:47.012 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-04-22T06:56:52.062 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-22T06:56:53.943 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T06:56:53.943 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T06:56:54.030 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-22T06:56:54.031 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-22T06:56:54.032 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-22T06:56:54.133 DEBUG:teuthology.orchestra.run.smithi092:> echo no | sudo yum reinstall kernel || true 2024-04-22T06:56:54.629 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:02 ago on Mon 22 Apr 2024 06:56:52 AM UTC. 2024-04-22T06:56:54.747 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-22T06:56:54.762 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-22T06:56:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-04-22T06:56:54.762 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-22T06:56:54.762 INFO:teuthology.orchestra.run.smithi092.stdout:Reinstalling: 2024-04-22T06:56:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:56:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T06:56:54.763 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-22T06:56:54.763 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-22T06:56:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T06:56:54.764 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 255 k 2024-04-22T06:56:54.764 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 0 2024-04-22T06:56:54.764 INFO:teuthology.orchestra.run.smithi092.stderr:Operation aborted. 2024-04-22T06:56:54.824 INFO:teuthology.orchestra.run.smithi092.stdout:Is this ok [y/N]: 2024-04-22T06:56:54.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum reinstall -y kernel || true 2024-04-22T06:56:55.319 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 06:56:52 AM UTC. 2024-04-22T06:56:55.434 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-22T06:56:55.435 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-22T06:56:55.435 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-04-22T06:56:55.435 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-22T06:56:55.435 INFO:teuthology.orchestra.run.smithi092.stdout:Reinstalling: 2024-04-22T06:56:55.436 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:56:55.436 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T06:56:55.436 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-22T06:56:55.436 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-22T06:56:55.436 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T06:56:55.436 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 255 k 2024-04-22T06:56:55.436 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 0 2024-04-22T06:56:55.436 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-22T06:56:55.805 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.7 MB/s | 255 kB 00:00 2024-04-22T06:56:55.805 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-22T06:56:55.806 INFO:teuthology.orchestra.run.smithi092.stdout:Total 689 kB/s | 255 kB 00:00 2024-04-22T06:56:55.827 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-22T06:56:56.001 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-22T06:56:56.001 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-22T06:56:56.006 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-22T06:56:56.006 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-22T06:56:56.156 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-22T06:56:56.222 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 799 kB/s | 6.2 MB 00:07 2024-04-22T06:56:56.222 INFO:teuthology.orchestra.run.smithi092.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:56:56.223 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:56:56.941 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:56:56.942 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:56:57.626 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:56:57.626 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T06:56:57.626 INFO:teuthology.orchestra.run.smithi092.stdout:Reinstalled: 2024-04-22T06:56:57.626 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T06:56:57.626 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T06:56:57.626 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-22T06:56:57.861 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T06:56:57.906 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-22T06:56:57.907 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T06:56:57.908 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T06:56:57.908 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T06:56:57.908 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-22T06:56:57.908 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T06:56:57.936 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-04-22T06:56:57.961 DEBUG:teuthology.parallel:result is None 2024-04-22T06:56:59.582 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-22T06:57:04.590 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-22T06:57:06.413 INFO:teuthology.orchestra.run.smithi145.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T06:57:06.413 INFO:teuthology.orchestra.run.smithi145.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T06:57:06.506 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-04-22T06:57:06.507 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2024-04-22T06:57:06.507 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-04-22T06:57:06.630 DEBUG:teuthology.orchestra.run.smithi145:> echo no | sudo yum reinstall kernel || true 2024-04-22T06:57:07.164 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 06:57:04 AM UTC. 2024-04-22T06:57:07.294 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-04-22T06:57:07.302 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-04-22T06:57:07.303 INFO:teuthology.orchestra.run.smithi145.stdout: Package Architecture Version Repository Size 2024-04-22T06:57:07.303 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-04-22T06:57:07.303 INFO:teuthology.orchestra.run.smithi145.stdout:Reinstalling: 2024-04-22T06:57:07.303 INFO:teuthology.orchestra.run.smithi145.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:57:07.303 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T06:57:07.303 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-04-22T06:57:07.303 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-04-22T06:57:07.303 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T06:57:07.304 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 255 k 2024-04-22T06:57:07.304 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 0 2024-04-22T06:57:07.304 INFO:teuthology.orchestra.run.smithi145.stderr:Operation aborted. 2024-04-22T06:57:07.361 INFO:teuthology.orchestra.run.smithi145.stdout:Is this ok [y/N]: 2024-04-22T06:57:07.361 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum reinstall -y kernel || true 2024-04-22T06:57:07.853 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 06:57:04 AM UTC. 2024-04-22T06:57:07.970 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-04-22T06:57:07.971 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-04-22T06:57:07.971 INFO:teuthology.orchestra.run.smithi145.stdout: Package Architecture Version Repository Size 2024-04-22T06:57:07.971 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-04-22T06:57:07.971 INFO:teuthology.orchestra.run.smithi145.stdout:Reinstalling: 2024-04-22T06:57:07.971 INFO:teuthology.orchestra.run.smithi145.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:57:07.971 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T06:57:07.971 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-04-22T06:57:07.971 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-04-22T06:57:07.971 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T06:57:07.971 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 255 k 2024-04-22T06:57:07.971 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 0 2024-04-22T06:57:07.972 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-04-22T06:57:08.351 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-5.14.0-437.el9.x86_64.rpm 967 kB/s | 255 kB 00:00 2024-04-22T06:57:08.351 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-04-22T06:57:08.351 INFO:teuthology.orchestra.run.smithi145.stdout:Total 671 kB/s | 255 kB 00:00 2024-04-22T06:57:08.368 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-04-22T06:57:08.566 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-04-22T06:57:08.566 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-04-22T06:57:08.571 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-04-22T06:57:08.571 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-04-22T06:57:08.710 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-04-22T06:57:08.785 INFO:teuthology.orchestra.run.smithi145.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:57:08.785 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:57:09.528 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:57:09.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:57:10.200 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:57:10.200 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T06:57:10.200 INFO:teuthology.orchestra.run.smithi145.stdout:Reinstalled: 2024-04-22T06:57:10.200 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T06:57:10.200 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T06:57:10.200 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-04-22T06:57:10.371 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T06:57:10.458 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T06:57:10.459 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T06:57:10.459 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T06:57:10.459 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-22T06:57:10.459 DEBUG:teuthology.orchestra.run.smithi145:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T06:57:10.492 INFO:teuthology.orchestra.run.smithi145.stdout:ttyS1 2024-04-22T06:57:10.522 DEBUG:teuthology.parallel:result is None 2024-04-22T06:57:10.523 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-22T06:57:10.530 INFO:teuthology.task.internal:Creating test directory... 2024-04-22T06:57:10.530 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T06:57:10.533 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T06:57:10.577 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-22T06:57:10.621 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-22T06:57:10.636 INFO:teuthology.task.internal:Creating archive directory... 2024-04-22T06:57:10.636 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T06:57:10.638 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T06:57:10.665 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-22T06:57:10.713 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-22T06:57:10.713 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T06:57:10.716 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T06:57:10.745 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:10.747 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:10.754 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:10.758 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:10.760 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-22T06:57:10.766 INFO:teuthology.task.internal:Configuring sudo... 2024-04-22T06:57:10.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T06:57:10.802 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T06:57:10.827 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-22T06:57:10.835 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-22T06:57:10.835 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T06:57:10.869 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T06:57:10.884 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:10.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:11.017 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:11.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:11.138 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T06:57:11.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T06:57:11.203 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:11.248 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:11.292 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:11.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:11.412 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T06:57:11.412 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T06:57:11.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-04-22T06:57:11.478 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2024-04-22T06:57:11.508 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T06:57:11.558 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T06:57:11.934 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-22T06:57:11.941 INFO:teuthology.task.internal:Starting timer... 2024-04-22T06:57:11.941 INFO:teuthology.run_tasks:Running task pcp... 2024-04-22T06:57:12.001 INFO:teuthology.run_tasks:Running task selinux... 2024-04-22T06:57:12.009 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-22T06:57:12.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-04-22T06:57:12.093 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-04-22T06:57:12.095 DEBUG:teuthology.orchestra.run.smithi145:> sudo service auditd rotate 2024-04-22T06:57:12.163 INFO:teuthology.orchestra.run.smithi145.stdout:Rotating logs: 2024-04-22T06:57:12.165 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-22T06:57:12.165 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-04-22T06:57:12.186 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-04-22T06:57:12.186 DEBUG:teuthology.orchestra.run.smithi145:> /usr/sbin/getenforce 2024-04-22T06:57:12.207 INFO:teuthology.orchestra.run.smithi145.stdout:Permissive 2024-04-22T06:57:12.208 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi145.front.sepia.ceph.com': 'permissive'} 2024-04-22T06:57:12.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T06:57:12.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:57:12.251 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T06:57:12.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:57:12.275 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-22T06:57:12.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-04-22T06:57:12.316 DEBUG:teuthology.orchestra.run.smithi145:> sudo /usr/sbin/setenforce permissive 2024-04-22T06:57:12.340 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-22T06:57:12.349 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-22T06:57:12.365 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-22T06:57:12.367 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi145.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-22T07:00:39.835 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi145.front.sepia.ceph.com')] 2024-04-22T07:00:39.837 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-04-22T07:00:39.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T07:00:39.923 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-04-22T07:00:40.001 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-04-22T07:00:40.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-04-22T07:00:40.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T07:00:40.089 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-04-22T07:00:40.178 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-04-22T07:00:40.179 INFO:teuthology.run_tasks:Running task clock... 2024-04-22T07:00:40.190 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-22T07:00:40.190 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T07:00:40.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:00:40.193 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T07:00:40.194 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:00:40.219 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T07:00:40.235 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T07:00:40.247 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T07:00:40.255 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-04-22T07:00:40.261 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T07:00:40.265 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-04-22T07:00:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T07:00:40.284 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: ntpd: command not found 2024-04-22T07:00:40.291 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T07:00:40.294 INFO:teuthology.orchestra.run.smithi145.stdout:506 Cannot talk to daemon 2024-04-22T07:00:40.308 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T07:00:40.321 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T07:00:40.347 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:00:40.351 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:00:40.351 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-04-22T07:00:40.351 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:40.351 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:40.351 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:40.351 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:40.378 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:00:40.382 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:00:40.382 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-04-22T07:00:40.383 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:40.383 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:40.383 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:40.383 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:40.383 INFO:teuthology.run_tasks:Running task install... 2024-04-22T07:00:40.392 DEBUG:teuthology.task.install:project ceph 2024-04-22T07:00:40.392 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}} 2024-04-22T07:00:40.392 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'} 2024-04-22T07:00:40.392 INFO:teuthology.task.install:Using flavor: default 2024-04-22T07:00:40.398 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-22T07:00:40.398 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-22T07:00:40.398 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'tag': None, 'wait_for_package': False} 2024-04-22T07:00:40.399 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:00:40.401 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'tag': None, 'wait_for_package': False} 2024-04-22T07:00:40.401 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:00:40.571 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/ 2024-04-22T07:00:40.572 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.g86f7587a 2024-04-22T07:00:40.638 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/ 2024-04-22T07:00:40.638 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.g86f7587a 2024-04-22T07:00:40.697 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T07:00:40.697 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:00:40.697 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T07:00:40.723 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T07:00:40.723 DEBUG:teuthology.orchestra.run.smithi145:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/86f7587a5a09af35f0895e1a2d08527638fae697/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T07:00:40.788 DEBUG:teuthology.orchestra.run.smithi145:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T07:00:40.818 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T07:00:40.819 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:00:40.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T07:00:40.844 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T07:00:40.845 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/86f7587a5a09af35f0895e1a2d08527638fae697/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T07:00:40.861 DEBUG:teuthology.orchestra.run.smithi145:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T07:00:40.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T07:00:40.924 INFO:teuthology.orchestra.run.smithi145.stdout:check_obsoletes = 1 2024-04-22T07:00:40.925 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum clean all 2024-04-22T07:00:40.985 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T07:00:41.047 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-04-22T07:00:41.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-04-22T07:00:41.147 INFO:teuthology.orchestra.run.smithi145.stdout:48 files removed 2024-04-22T07:00:41.175 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T07:00:41.272 INFO:teuthology.orchestra.run.smithi092.stdout:48 files removed 2024-04-22T07:00:41.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T07:00:41.849 INFO:teuthology.orchestra.run.smithi145.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-04-22T07:00:41.907 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 238 kB/s | 83 kB 00:00 2024-04-22T07:00:42.274 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-22T07:00:42.308 INFO:teuthology.orchestra.run.smithi145.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-04-22T07:00:42.650 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-04-22T07:00:42.725 INFO:teuthology.orchestra.run.smithi145.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-22T07:00:43.183 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-22T07:00:51.641 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 928 kB/s | 8.0 MB 00:08 2024-04-22T07:00:57.383 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 19 MB 00:13 2024-04-22T07:01:01.491 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-22T07:01:03.009 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 16 kB 00:00 2024-04-22T07:01:06.447 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 21 MB 00:03 2024-04-22T07:01:09.357 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-04-22T07:01:11.559 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-22T07:01:13.694 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:01:13.694 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:01:13.705 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T07:01:13.705 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T07:01:13.705 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T07:01:13.772 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-22T07:01:13.778 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-04-22T07:01:13.778 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-22T07:01:13.779 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-04-22T07:01:13.779 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-22T07:01:13.779 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T07:01:13.779 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 6.6 k 2024-04-22T07:01:13.779 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:01:13.779 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 890 k 2024-04-22T07:01:13.779 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 145 k 2024-04-22T07:01:13.779 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 1.5 M 2024-04-22T07:01:13.779 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 142 k 2024-04-22T07:01:13.779 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 3.5 M 2024-04-22T07:01:13.779 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 7.4 M 2024-04-22T07:01:13.779 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 49 k 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 11 M 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 49 M 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 273 k 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 763 k 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 225 k 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 27 k 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 18 k 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 32 k 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 747 k 2024-04-22T07:01:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 127 k 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 162 k 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 320 k 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 298 k 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 100 k 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 85 k 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:01:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 171 k 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.4 M 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 21 M 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 34 k 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 2.4 M 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 249 k 2024-04-22T07:01:13.782 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 4.7 M 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 17 M 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 16 k 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 25 k 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 164 k 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T07:01:13.783 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 500 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T07:01:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 45 k 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 135 k 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T07:01:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T07:01:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T07:01:13.787 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T07:01:13.788 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T07:01:13.789 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T07:01:13.790 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T07:01:13.790 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T07:01:13.790 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T07:01:13.790 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-04-22T07:01:13.790 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T07:01:13.790 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:01:13.790 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-22T07:01:13.790 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-04-22T07:01:13.790 INFO:teuthology.orchestra.run.smithi092.stdout:Install 148 Packages 2024-04-22T07:01:13.790 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2024-04-22T07:01:13.790 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:01:13.791 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 218 M 2024-04-22T07:01:13.791 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-22T07:01:19.965 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 898 kB/s | 6.2 MB 00:07 2024-04-22T07:01:23.306 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 7.3 kB/s | 16 kB 00:02 2024-04-22T07:01:23.309 INFO:teuthology.orchestra.run.smithi092.stdout:(1/150): ceph-19.0.0-3156.g86f7587a.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-22T07:01:24.567 INFO:teuthology.orchestra.run.smithi092.stdout:(2/150): ceph-fuse-19.0.0-3156.g86f7587a.el9.x8 708 kB/s | 890 kB 00:01 2024-04-22T07:01:24.826 INFO:teuthology.orchestra.run.smithi092.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 561 kB/s | 145 kB 00:00 2024-04-22T07:01:26.817 INFO:teuthology.orchestra.run.smithi092.stdout:(4/150): ceph-mds-19.0.0-3156.g86f7587a.el9.x86 1.2 MB/s | 2.4 MB 00:01 2024-04-22T07:01:26.874 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 6.1 MB/s | 21 MB 00:03 2024-04-22T07:01:28.226 INFO:teuthology.orchestra.run.smithi092.stdout:(5/150): ceph-base-19.0.0-3156.g86f7587a.el9.x8 1.0 MB/s | 5.4 MB 00:05 2024-04-22T07:01:28.267 INFO:teuthology.orchestra.run.smithi092.stdout:(6/150): ceph-mgr-19.0.0-3156.g86f7587a.el9.x86 1.0 MB/s | 1.5 MB 00:01 2024-04-22T07:01:31.965 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-22T07:01:32.642 INFO:teuthology.orchestra.run.smithi092.stdout:(7/150): ceph-mon-19.0.0-3156.g86f7587a.el9.x86 1.1 MB/s | 4.7 MB 00:04 2024-04-22T07:01:34.108 INFO:teuthology.orchestra.run.smithi145.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:01:34.109 INFO:teuthology.orchestra.run.smithi145.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:01:34.119 INFO:teuthology.orchestra.run.smithi145.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T07:01:34.120 INFO:teuthology.orchestra.run.smithi145.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T07:01:34.121 INFO:teuthology.orchestra.run.smithi145.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T07:01:34.121 INFO:teuthology.orchestra.run.smithi145.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T07:01:34.123 INFO:teuthology.orchestra.run.smithi145.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T07:01:34.125 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T07:01:34.126 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T07:01:34.185 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-04-22T07:01:34.194 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================= 2024-04-22T07:01:34.194 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-04-22T07:01:34.194 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================= 2024-04-22T07:01:34.194 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-04-22T07:01:34.194 INFO:teuthology.orchestra.run.smithi145.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T07:01:34.194 INFO:teuthology.orchestra.run.smithi145.stdout: ceph x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 6.6 k 2024-04-22T07:01:34.194 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:01:34.194 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 890 k 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 145 k 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 1.5 M 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 142 k 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 3.5 M 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 7.4 M 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 49 k 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 11 M 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 49 M 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 273 k 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 763 k 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 225 k 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 27 k 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 18 k 2024-04-22T07:01:34.195 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 32 k 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 747 k 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 127 k 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 162 k 2024-04-22T07:01:34.196 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 320 k 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 298 k 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 100 k 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 85 k 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 171 k 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrading: 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout: librados2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.4 M 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T07:01:34.197 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 21 M 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 34 k 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 2.4 M 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 249 k 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 4.7 M 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 17 M 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 16 k 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 25 k 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T07:01:34.198 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 164 k 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 500 k 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:01:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T07:01:34.200 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 45 k 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 135 k 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T07:01:34.201 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T07:01:34.202 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T07:01:34.203 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T07:01:34.204 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout:Installing weak dependencies: 2024-04-22T07:01:34.205 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T07:01:34.206 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:01:34.206 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-04-22T07:01:34.206 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================= 2024-04-22T07:01:34.206 INFO:teuthology.orchestra.run.smithi145.stdout:Install 148 Packages 2024-04-22T07:01:34.206 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrade 2 Packages 2024-04-22T07:01:34.206 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:01:34.206 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 218 M 2024-04-22T07:01:34.207 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-04-22T07:01:37.108 INFO:teuthology.orchestra.run.smithi092.stdout:(8/150): ceph-osd-19.0.0-3156.g86f7587a.el9.x86 1.9 MB/s | 17 MB 00:08 2024-04-22T07:01:37.175 INFO:teuthology.orchestra.run.smithi092.stdout:(9/150): ceph-selinux-19.0.0-3156.g86f7587a.el9 380 kB/s | 25 kB 00:00 2024-04-22T07:01:37.692 INFO:teuthology.orchestra.run.smithi092.stdout:(10/150): ceph-common-19.0.0-3156.g86f7587a.el9 1.4 MB/s | 21 MB 00:14 2024-04-22T07:01:37.934 INFO:teuthology.orchestra.run.smithi092.stdout:(11/150): cephfs-mirror-19.0.0-3156.g86f7587a.e 931 kB/s | 225 kB 00:00 2024-04-22T07:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout:(12/150): cephfs-shell-19.0.0-3156.g86f7587a.el 325 kB/s | 27 kB 00:00 2024-04-22T07:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout:(13/150): libcephfs-devel-19.0.0-3156.g86f7587a 346 kB/s | 32 kB 00:00 2024-04-22T07:01:38.526 INFO:teuthology.orchestra.run.smithi092.stdout:(14/150): ceph-radosgw-19.0.0-3156.g86f7587a.el 1.8 MB/s | 11 MB 00:05 2024-04-22T07:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout:(15/150): libcephfs2-19.0.0-3156.g86f7587a.el9. 1.5 MB/s | 747 kB 00:00 2024-04-22T07:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout:(16/150): libcephsqlite-19.0.0-3156.g86f7587a.e 1.3 MB/s | 164 kB 00:00 2024-04-22T07:01:38.658 INFO:teuthology.orchestra.run.smithi145.stdout:(1/150): ceph-19.0.0-3156.g86f7587a.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-22T07:01:38.685 INFO:teuthology.orchestra.run.smithi092.stdout:(17/150): librados-devel-19.0.0-3156.g86f7587a. 1.5 MB/s | 127 kB 00:00 2024-04-22T07:01:38.944 INFO:teuthology.orchestra.run.smithi092.stdout:(18/150): libradosstriper1-19.0.0-3156.g86f7587 1.7 MB/s | 500 kB 00:00 2024-04-22T07:01:39.036 INFO:teuthology.orchestra.run.smithi092.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.g86 493 kB/s | 45 kB 00:00 2024-04-22T07:01:39.128 INFO:teuthology.orchestra.run.smithi092.stdout:(20/150): python3-ceph-common-19.0.0-3156.g86f7 1.4 MB/s | 135 kB 00:00 2024-04-22T07:01:39.287 INFO:teuthology.orchestra.run.smithi092.stdout:(21/150): python3-cephfs-19.0.0-3156.g86f7587a. 1.0 MB/s | 162 kB 00:00 2024-04-22T07:01:39.470 INFO:teuthology.orchestra.run.smithi092.stdout:(22/150): python3-rados-19.0.0-3156.g86f7587a.e 1.7 MB/s | 320 kB 00:00 2024-04-22T07:01:39.654 INFO:teuthology.orchestra.run.smithi092.stdout:(23/150): python3-rbd-19.0.0-3156.g86f7587a.el9 1.6 MB/s | 298 kB 00:00 2024-04-22T07:01:39.754 INFO:teuthology.orchestra.run.smithi092.stdout:(24/150): python3-rgw-19.0.0-3156.g86f7587a.el9 1.0 MB/s | 100 kB 00:00 2024-04-22T07:01:39.838 INFO:teuthology.orchestra.run.smithi092.stdout:(25/150): rbd-fuse-19.0.0-3156.g86f7587a.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-22T07:01:40.052 INFO:teuthology.orchestra.run.smithi145.stdout:(2/150): ceph-fuse-19.0.0-3156.g86f7587a.el9.x8 639 kB/s | 890 kB 00:01 2024-04-22T07:01:40.142 INFO:teuthology.orchestra.run.smithi145.stdout:(3/150): ceph-base-19.0.0-3156.g86f7587a.el9.x8 2.9 MB/s | 5.4 MB 00:01 2024-04-22T07:01:40.300 INFO:teuthology.orchestra.run.smithi145.stdout:(4/150): ceph-immutable-object-cache-19.0.0-315 584 kB/s | 145 kB 00:00 2024-04-22T07:01:40.625 INFO:teuthology.orchestra.run.smithi145.stdout:(5/150): ceph-mds-19.0.0-3156.g86f7587a.el9.x86 4.9 MB/s | 2.4 MB 00:00 2024-04-22T07:01:41.155 INFO:teuthology.orchestra.run.smithi092.stdout:(26/150): rbd-mirror-19.0.0-3156.g86f7587a.el9. 2.3 MB/s | 3.1 MB 00:01 2024-04-22T07:01:41.305 INFO:teuthology.orchestra.run.smithi092.stdout:(27/150): rbd-nbd-19.0.0-3156.g86f7587a.el9.x86 1.1 MB/s | 171 kB 00:00 2024-04-22T07:01:41.405 INFO:teuthology.orchestra.run.smithi092.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3156.g 342 kB/s | 34 kB 00:00 2024-04-22T07:01:41.497 INFO:teuthology.orchestra.run.smithi092.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3156.g86f7587 1.5 MB/s | 142 kB 00:00 2024-04-22T07:01:41.542 INFO:teuthology.orchestra.run.smithi145.stdout:(6/150): ceph-mon-19.0.0-3156.g86f7587a.el9.x86 5.1 MB/s | 4.7 MB 00:00 2024-04-22T07:01:42.376 INFO:teuthology.orchestra.run.smithi145.stdout:(7/150): ceph-mgr-19.0.0-3156.g86f7587a.el9.x86 737 kB/s | 1.5 MB 00:02 2024-04-22T07:01:42.447 INFO:teuthology.orchestra.run.smithi092.stdout:(30/150): librgw2-19.0.0-3156.g86f7587a.el9.x86 1.4 MB/s | 5.4 MB 00:03 2024-04-22T07:01:43.014 INFO:teuthology.orchestra.run.smithi092.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3156.g86f75 2.3 MB/s | 3.5 MB 00:01 2024-04-22T07:01:43.181 INFO:teuthology.orchestra.run.smithi092.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3156.g86 1.5 MB/s | 249 kB 00:00 2024-04-22T07:01:43.273 INFO:teuthology.orchestra.run.smithi092.stdout:(33/150): ceph-mgr-rook-19.0.0-3156.g86f7587a.e 539 kB/s | 49 kB 00:00 2024-04-22T07:01:43.340 INFO:teuthology.orchestra.run.smithi092.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3156.g8 246 kB/s | 16 kB 00:00 2024-04-22T07:01:43.457 INFO:teuthology.orchestra.run.smithi092.stdout:(35/150): ceph-volume-19.0.0-3156.g86f7587a.el9 2.3 MB/s | 273 kB 00:00 2024-04-22T07:01:43.774 INFO:teuthology.orchestra.run.smithi092.stdout:(36/150): cephadm-19.0.0-3156.g86f7587a.el9.noa 2.4 MB/s | 763 kB 00:00 2024-04-22T07:01:43.849 INFO:teuthology.orchestra.run.smithi092.stdout:(37/150): cephfs-top-19.0.0-3156.g86f7587a.el9. 244 kB/s | 18 kB 00:00 2024-04-22T07:01:43.957 INFO:teuthology.orchestra.run.smithi092.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 741 kB/s | 80 kB 00:00 2024-04-22T07:01:44.066 INFO:teuthology.orchestra.run.smithi092.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-04-22T07:01:44.108 INFO:teuthology.orchestra.run.smithi092.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-22T07:01:44.150 INFO:teuthology.orchestra.run.smithi092.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-22T07:01:44.217 INFO:teuthology.orchestra.run.smithi092.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-22T07:01:44.250 INFO:teuthology.orchestra.run.smithi092.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-22T07:01:44.292 INFO:teuthology.orchestra.run.smithi092.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-22T07:01:44.351 INFO:teuthology.orchestra.run.smithi092.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-22T07:01:44.394 INFO:teuthology.orchestra.run.smithi092.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-04-22T07:01:44.494 INFO:teuthology.orchestra.run.smithi092.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-04-22T07:01:44.602 INFO:teuthology.orchestra.run.smithi092.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-04-22T07:01:44.683 INFO:teuthology.orchestra.run.smithi092.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-04-22T07:01:44.744 INFO:teuthology.orchestra.run.smithi092.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 356 kB/s | 21 kB 00:00 2024-04-22T07:01:44.822 INFO:teuthology.orchestra.run.smithi092.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-04-22T07:01:44.881 INFO:teuthology.orchestra.run.smithi092.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-04-22T07:01:45.062 INFO:teuthology.orchestra.run.smithi145.stdout:(8/150): ceph-osd-19.0.0-3156.g86f7587a.el9.x86 4.8 MB/s | 17 MB 00:03 2024-04-22T07:01:45.131 INFO:teuthology.orchestra.run.smithi092.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-04-22T07:01:45.168 INFO:teuthology.orchestra.run.smithi145.stdout:(9/150): ceph-selinux-19.0.0-3156.g86f7587a.el9 233 kB/s | 25 kB 00:00 2024-04-22T07:01:45.172 INFO:teuthology.orchestra.run.smithi092.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-04-22T07:01:45.265 INFO:teuthology.orchestra.run.smithi092.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 120 kB/s | 11 kB 00:00 2024-04-22T07:01:45.358 INFO:teuthology.orchestra.run.smithi092.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-04-22T07:01:45.415 INFO:teuthology.orchestra.run.smithi092.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 778 kB/s | 45 kB 00:00 2024-04-22T07:01:45.540 INFO:teuthology.orchestra.run.smithi092.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-04-22T07:01:45.657 INFO:teuthology.orchestra.run.smithi092.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-04-22T07:01:45.699 INFO:teuthology.orchestra.run.smithi092.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 613 kB/s | 25 kB 00:00 2024-04-22T07:01:45.808 INFO:teuthology.orchestra.run.smithi092.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-04-22T07:01:45.908 INFO:teuthology.orchestra.run.smithi092.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-04-22T07:01:45.983 INFO:teuthology.orchestra.run.smithi092.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-22T07:01:46.042 INFO:teuthology.orchestra.run.smithi092.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 604 kB/s | 35 kB 00:00 2024-04-22T07:01:46.459 INFO:teuthology.orchestra.run.smithi092.stdout:(65/150): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-04-22T07:01:46.717 INFO:teuthology.orchestra.run.smithi092.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 16 MB/s | 4.1 MB 00:00 2024-04-22T07:01:46.801 INFO:teuthology.orchestra.run.smithi092.stdout:(67/150): ceph-mgr-diskprediction-local-19.0.0- 1.7 MB/s | 7.4 MB 00:04 2024-04-22T07:01:46.935 INFO:teuthology.orchestra.run.smithi092.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-04-22T07:01:46.976 INFO:teuthology.orchestra.run.smithi092.stdout:(69/150): postgresql-13.14-1.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-04-22T07:01:47.018 INFO:teuthology.orchestra.run.smithi092.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-22T07:01:47.202 INFO:teuthology.orchestra.run.smithi092.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-04-22T07:01:47.269 INFO:teuthology.orchestra.run.smithi092.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-22T07:01:47.386 INFO:teuthology.orchestra.run.smithi092.stdout:(73/150): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-04-22T07:01:47.469 INFO:teuthology.orchestra.run.smithi092.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-22T07:01:47.562 INFO:teuthology.orchestra.run.smithi092.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-22T07:01:47.620 INFO:teuthology.orchestra.run.smithi092.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-04-22T07:01:47.695 INFO:teuthology.orchestra.run.smithi092.stdout:(77/150): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-04-22T07:01:47.746 INFO:teuthology.orchestra.run.smithi092.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 152 kB/s | 19 kB 00:00 2024-04-22T07:01:47.788 INFO:teuthology.orchestra.run.smithi092.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-22T07:01:47.813 INFO:teuthology.orchestra.run.smithi092.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 518 kB/s | 35 kB 00:00 2024-04-22T07:01:47.888 INFO:teuthology.orchestra.run.smithi092.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-22T07:01:47.972 INFO:teuthology.orchestra.run.smithi092.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-22T07:01:48.064 INFO:teuthology.orchestra.run.smithi092.stdout:(83/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-04-22T07:01:48.089 INFO:teuthology.orchestra.run.smithi092.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 661 kB/s | 77 kB 00:00 2024-04-22T07:01:48.139 INFO:teuthology.orchestra.run.smithi092.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 563 kB/s | 42 kB 00:00 2024-04-22T07:01:48.181 INFO:teuthology.orchestra.run.smithi092.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-22T07:01:48.223 INFO:teuthology.orchestra.run.smithi092.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-22T07:01:48.290 INFO:teuthology.orchestra.run.smithi092.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 765 kB/s | 51 kB 00:00 2024-04-22T07:01:48.315 INFO:teuthology.orchestra.run.smithi092.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-04-22T07:01:48.399 INFO:teuthology.orchestra.run.smithi092.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 494 kB/s | 54 kB 00:00 2024-04-22T07:01:48.458 INFO:teuthology.orchestra.run.smithi092.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-04-22T07:01:48.517 INFO:teuthology.orchestra.run.smithi092.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-04-22T07:01:48.592 INFO:teuthology.orchestra.run.smithi092.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-22T07:01:48.634 INFO:teuthology.orchestra.run.smithi092.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 36 kB 00:00 2024-04-22T07:01:48.684 INFO:teuthology.orchestra.run.smithi092.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-04-22T07:01:48.818 INFO:teuthology.orchestra.run.smithi092.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-04-22T07:01:48.876 INFO:teuthology.orchestra.run.smithi092.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 997 kB/s | 58 kB 00:00 2024-04-22T07:01:48.952 INFO:teuthology.orchestra.run.smithi092.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-22T07:01:49.044 INFO:teuthology.orchestra.run.smithi092.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 695 kB/s | 64 kB 00:00 2024-04-22T07:01:49.135 INFO:teuthology.orchestra.run.smithi092.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 491 kB/s | 45 kB 00:00 2024-04-22T07:01:49.286 INFO:teuthology.orchestra.run.smithi092.stdout:(101/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-22T07:01:49.361 INFO:teuthology.orchestra.run.smithi092.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 99 kB/s | 22 kB 00:00 2024-04-22T07:01:49.644 INFO:teuthology.orchestra.run.smithi092.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-04-22T07:01:50.501 INFO:teuthology.orchestra.run.smithi145.stdout:(10/150): ceph-common-19.0.0-3156.g86f7587a.el9 1.7 MB/s | 21 MB 00:12 2024-04-22T07:01:50.578 INFO:teuthology.orchestra.run.smithi092.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 331 kB/s | 309 kB 00:00 2024-04-22T07:01:50.617 INFO:teuthology.orchestra.run.smithi145.stdout:(11/150): cephfs-mirror-19.0.0-3156.g86f7587a.e 1.9 MB/s | 225 kB 00:00 2024-04-22T07:01:50.676 INFO:teuthology.orchestra.run.smithi145.stdout:(12/150): cephfs-shell-19.0.0-3156.g86f7587a.el 464 kB/s | 27 kB 00:00 2024-04-22T07:01:50.695 INFO:teuthology.orchestra.run.smithi092.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-04-22T07:01:50.726 INFO:teuthology.orchestra.run.smithi145.stdout:(13/150): libcephfs-devel-19.0.0-3156.g86f7587a 636 kB/s | 32 kB 00:00 2024-04-22T07:01:50.728 INFO:teuthology.orchestra.run.smithi092.stdout:(106/150): abseil-cpp-20211102.0-3.el9.x86_64.r 401 kB/s | 548 kB 00:01 2024-04-22T07:01:50.845 INFO:teuthology.orchestra.run.smithi092.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-04-22T07:01:50.962 INFO:teuthology.orchestra.run.smithi092.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-04-22T07:01:50.968 INFO:teuthology.orchestra.run.smithi145.stdout:(14/150): libcephfs2-19.0.0-3156.g86f7587a.el9. 3.0 MB/s | 747 kB 00:00 2024-04-22T07:01:51.035 INFO:teuthology.orchestra.run.smithi145.stdout:(15/150): libcephsqlite-19.0.0-3156.g86f7587a.e 2.4 MB/s | 164 kB 00:00 2024-04-22T07:01:51.102 INFO:teuthology.orchestra.run.smithi145.stdout:(16/150): librados-devel-19.0.0-3156.g86f7587a. 1.9 MB/s | 127 kB 00:00 2024-04-22T07:01:51.171 INFO:teuthology.orchestra.run.smithi092.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 323 kB/s | 67 kB 00:00 2024-04-22T07:01:51.252 INFO:teuthology.orchestra.run.smithi145.stdout:(17/150): libradosstriper1-19.0.0-3156.g86f7587 3.3 MB/s | 500 kB 00:00 2024-04-22T07:01:51.428 INFO:teuthology.orchestra.run.smithi145.stdout:(18/150): ceph-radosgw-19.0.0-3156.g86f7587a.el 1.2 MB/s | 11 MB 00:09 2024-04-22T07:01:51.463 INFO:teuthology.orchestra.run.smithi092.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 519 kB/s | 151 kB 00:00 2024-04-22T07:01:51.562 INFO:teuthology.orchestra.run.smithi145.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.g86 339 kB/s | 45 kB 00:00 2024-04-22T07:01:51.745 INFO:teuthology.orchestra.run.smithi145.stdout:(20/150): python3-ceph-common-19.0.0-3156.g86f7 735 kB/s | 135 kB 00:00 2024-04-22T07:01:51.912 INFO:teuthology.orchestra.run.smithi145.stdout:(21/150): python3-cephfs-19.0.0-3156.g86f7587a. 972 kB/s | 162 kB 00:00 2024-04-22T07:01:52.162 INFO:teuthology.orchestra.run.smithi145.stdout:(22/150): python3-rados-19.0.0-3156.g86f7587a.e 1.3 MB/s | 320 kB 00:00 2024-04-22T07:01:52.429 INFO:teuthology.orchestra.run.smithi145.stdout:(23/150): python3-rbd-19.0.0-3156.g86f7587a.el9 1.1 MB/s | 298 kB 00:00 2024-04-22T07:01:52.505 INFO:teuthology.orchestra.run.smithi145.stdout:(24/150): librgw2-19.0.0-3156.g86f7587a.el9.x86 4.3 MB/s | 5.4 MB 00:01 2024-04-22T07:01:52.538 INFO:teuthology.orchestra.run.smithi145.stdout:(25/150): python3-rgw-19.0.0-3156.g86f7587a.el9 918 kB/s | 100 kB 00:00 2024-04-22T07:01:52.605 INFO:teuthology.orchestra.run.smithi145.stdout:(26/150): rbd-fuse-19.0.0-3156.g86f7587a.el9.x8 852 kB/s | 85 kB 00:00 2024-04-22T07:01:52.714 INFO:teuthology.orchestra.run.smithi145.stdout:(27/150): rbd-nbd-19.0.0-3156.g86f7587a.el9.x86 1.5 MB/s | 171 kB 00:00 2024-04-22T07:01:52.755 INFO:teuthology.orchestra.run.smithi092.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 649 kB/s | 838 kB 00:01 2024-04-22T07:01:52.814 INFO:teuthology.orchestra.run.smithi145.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3156.g 342 kB/s | 34 kB 00:00 2024-04-22T07:01:52.898 INFO:teuthology.orchestra.run.smithi145.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3156.g86f7587 1.7 MB/s | 142 kB 00:00 2024-04-22T07:01:53.565 INFO:teuthology.orchestra.run.smithi145.stdout:(30/150): ceph-mgr-dashboard-19.0.0-3156.g86f75 5.3 MB/s | 3.5 MB 00:00 2024-04-22T07:01:53.588 INFO:teuthology.orchestra.run.smithi092.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 658 kB/s | 548 kB 00:00 2024-04-22T07:01:53.713 INFO:teuthology.orchestra.run.smithi092.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 348 kB/s | 43 kB 00:00 2024-04-22T07:01:53.830 INFO:teuthology.orchestra.run.smithi092.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2024-04-22T07:01:53.947 INFO:teuthology.orchestra.run.smithi092.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2024-04-22T07:01:54.248 INFO:teuthology.orchestra.run.smithi092.stdout:(116/150): python3-cheroot-8.6.0-4.el9.noarch.r 573 kB/s | 172 kB 00:00 2024-04-22T07:01:54.348 INFO:teuthology.orchestra.run.smithi145.stdout:(31/150): rbd-mirror-19.0.0-3156.g86f7587a.el9. 1.7 MB/s | 3.1 MB 00:01 2024-04-22T07:01:54.515 INFO:teuthology.orchestra.run.smithi145.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3156.g86 1.5 MB/s | 249 kB 00:00 2024-04-22T07:01:54.658 INFO:teuthology.orchestra.run.smithi145.stdout:(33/150): ceph-mgr-diskprediction-local-19.0.0- 6.8 MB/s | 7.4 MB 00:01 2024-04-22T07:01:54.682 INFO:teuthology.orchestra.run.smithi145.stdout:(34/150): ceph-mgr-rook-19.0.0-3156.g86f7587a.e 296 kB/s | 49 kB 00:00 2024-04-22T07:01:54.716 INFO:teuthology.orchestra.run.smithi145.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3156.g8 282 kB/s | 16 kB 00:00 2024-04-22T07:01:54.723 INFO:teuthology.orchestra.run.smithi092.stdout:(117/150): ceph-test-19.0.0-3156.g86f7587a.el9. 2.8 MB/s | 49 MB 00:17 2024-04-22T07:01:54.931 INFO:teuthology.orchestra.run.smithi092.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 525 kB/s | 358 kB 00:00 2024-04-22T07:01:55.048 INFO:teuthology.orchestra.run.smithi092.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-22T07:01:55.150 INFO:teuthology.orchestra.run.smithi145.stdout:(36/150): ceph-test-19.0.0-3156.g86f7587a.el9.x 4.9 MB/s | 49 MB 00:09 2024-04-22T07:01:55.175 INFO:teuthology.orchestra.run.smithi145.stdout:(37/150): ceph-volume-19.0.0-3156.g86f7587a.el9 555 kB/s | 273 kB 00:00 2024-04-22T07:01:55.200 INFO:teuthology.orchestra.run.smithi145.stdout:(38/150): cephfs-top-19.0.0-3156.g86f7587a.el9. 366 kB/s | 18 kB 00:00 2024-04-22T07:01:55.242 INFO:teuthology.orchestra.run.smithi145.stdout:(39/150): cephadm-19.0.0-3156.g86f7587a.el9.noa 1.4 MB/s | 763 kB 00:00 2024-04-22T07:01:55.326 INFO:teuthology.orchestra.run.smithi145.stdout:(40/150): fuse-2.9.9-15.el9.x86_64.rpm 534 kB/s | 80 kB 00:00 2024-04-22T07:01:55.340 INFO:teuthology.orchestra.run.smithi092.stdout:(120/150): python3-google-auth-2.28.2-1.el9.noa 695 kB/s | 203 kB 00:00 2024-04-22T07:01:55.393 INFO:teuthology.orchestra.run.smithi145.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-04-22T07:01:55.418 INFO:teuthology.orchestra.run.smithi145.stdout:(42/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-22T07:01:55.443 INFO:teuthology.orchestra.run.smithi145.stdout:(43/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-22T07:01:55.485 INFO:teuthology.orchestra.run.smithi145.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-22T07:01:55.519 INFO:teuthology.orchestra.run.smithi145.stdout:(45/150): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-22T07:01:55.544 INFO:teuthology.orchestra.run.smithi145.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-22T07:01:55.557 INFO:teuthology.orchestra.run.smithi092.stdout:(121/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 364 kB/s | 303 kB 00:00 2024-04-22T07:01:55.578 INFO:teuthology.orchestra.run.smithi145.stdout:(47/150): python3-cryptography-36.0.1-4.el9.x86 6.6 MB/s | 1.2 MB 00:00 2024-04-22T07:01:55.603 INFO:teuthology.orchestra.run.smithi145.stdout:(48/150): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-22T07:01:55.670 INFO:teuthology.orchestra.run.smithi145.stdout:(49/150): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-04-22T07:01:55.695 INFO:teuthology.orchestra.run.smithi145.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-22T07:01:55.745 INFO:teuthology.orchestra.run.smithi145.stdout:(51/150): bison-3.7.4-5.el9.x86_64.rpm 5.5 MB/s | 943 kB 00:00 2024-04-22T07:01:55.771 INFO:teuthology.orchestra.run.smithi145.stdout:(52/150): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-04-22T07:01:55.812 INFO:teuthology.orchestra.run.smithi145.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-04-22T07:01:55.838 INFO:teuthology.orchestra.run.smithi145.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-22T07:01:55.849 INFO:teuthology.orchestra.run.smithi092.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 495 kB/s | 144 kB 00:00 2024-04-22T07:01:55.888 INFO:teuthology.orchestra.run.smithi145.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 242 kB/s | 18 kB 00:00 2024-04-22T07:01:55.913 INFO:teuthology.orchestra.run.smithi145.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-04-22T07:01:55.966 INFO:teuthology.orchestra.run.smithi092.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-04-22T07:01:56.022 INFO:teuthology.orchestra.run.smithi145.stdout:(57/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-04-22T07:01:56.047 INFO:teuthology.orchestra.run.smithi145.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-04-22T07:01:56.074 INFO:teuthology.orchestra.run.smithi092.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-22T07:01:56.089 INFO:teuthology.orchestra.run.smithi145.stdout:(59/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 798 kB/s | 160 kB 00:00 2024-04-22T07:01:56.139 INFO:teuthology.orchestra.run.smithi145.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 512 kB/s | 25 kB 00:00 2024-04-22T07:01:56.165 INFO:teuthology.orchestra.run.smithi145.stdout:(61/150): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-04-22T07:01:56.183 INFO:teuthology.orchestra.run.smithi092.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-04-22T07:01:56.240 INFO:teuthology.orchestra.run.smithi145.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-04-22T07:01:56.266 INFO:teuthology.orchestra.run.smithi145.stdout:(63/150): libtool-2.4.6-45.el9.x86_64.rpm 2.6 MB/s | 582 kB 00:00 2024-04-22T07:01:56.291 INFO:teuthology.orchestra.run.smithi092.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-04-22T07:01:56.299 INFO:teuthology.orchestra.run.smithi145.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-22T07:01:56.341 INFO:teuthology.orchestra.run.smithi145.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 35 kB 00:00 2024-04-22T07:01:56.366 INFO:teuthology.orchestra.run.smithi145.stdout:(66/150): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-22T07:01:56.436 INFO:teuthology.orchestra.run.smithi092.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 137 kB/s | 20 kB 00:00 2024-04-22T07:01:56.583 INFO:teuthology.orchestra.run.smithi145.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-22T07:01:56.666 INFO:teuthology.orchestra.run.smithi145.stdout:(68/150): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-04-22T07:01:56.758 INFO:teuthology.orchestra.run.smithi145.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-22T07:01:56.834 INFO:teuthology.orchestra.run.smithi092.stdout:(128/150): libarrow-9.0.0-10.el9.x86_64.rpm 737 kB/s | 4.4 MB 00:06 2024-04-22T07:01:56.842 INFO:teuthology.orchestra.run.smithi145.stdout:(70/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 8.3 MB/s | 4.1 MB 00:00 2024-04-22T07:01:56.876 INFO:teuthology.orchestra.run.smithi145.stdout:(71/150): postgresql-contrib-13.14-1.el9.x86_64 3.0 MB/s | 911 kB 00:00 2024-04-22T07:01:56.942 INFO:teuthology.orchestra.run.smithi145.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-22T07:01:56.943 INFO:teuthology.orchestra.run.smithi092.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 428 kB/s | 46 kB 00:00 2024-04-22T07:01:57.067 INFO:teuthology.orchestra.run.smithi092.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 631 kB/s | 79 kB 00:00 2024-04-22T07:01:57.068 INFO:teuthology.orchestra.run.smithi145.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-04-22T07:01:57.152 INFO:teuthology.orchestra.run.smithi145.stdout:(74/150): postgresql-server-13.14-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-04-22T07:01:57.176 INFO:teuthology.orchestra.run.smithi092.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 532 kB/s | 58 kB 00:00 2024-04-22T07:01:57.204 INFO:teuthology.orchestra.run.smithi145.stdout:(75/150): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-04-22T07:01:57.254 INFO:teuthology.orchestra.run.smithi145.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-22T07:01:57.271 INFO:teuthology.orchestra.run.smithi145.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 382 kB/s | 26 kB 00:00 2024-04-22T07:01:57.305 INFO:teuthology.orchestra.run.smithi145.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-04-22T07:01:57.338 INFO:teuthology.orchestra.run.smithi145.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-22T07:01:57.363 INFO:teuthology.orchestra.run.smithi145.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-04-22T07:01:57.439 INFO:teuthology.orchestra.run.smithi145.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-22T07:01:57.476 INFO:teuthology.orchestra.run.smithi092.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 908 kB/s | 272 kB 00:00 2024-04-22T07:01:57.539 INFO:teuthology.orchestra.run.smithi145.stdout:(82/150): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-22T07:01:57.585 INFO:teuthology.orchestra.run.smithi092.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-22T07:01:57.589 INFO:teuthology.orchestra.run.smithi145.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-04-22T07:01:57.615 INFO:teuthology.orchestra.run.smithi145.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-22T07:01:57.665 INFO:teuthology.orchestra.run.smithi145.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-04-22T07:01:57.702 INFO:teuthology.orchestra.run.smithi092.stdout:(134/150): python3-grpcio-1.46.7-10.el9.x86_64. 884 kB/s | 2.0 MB 00:02 2024-04-22T07:01:57.727 INFO:teuthology.orchestra.run.smithi092.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 634 kB/s | 90 kB 00:00 2024-04-22T07:01:57.731 INFO:teuthology.orchestra.run.smithi145.stdout:(86/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-22T07:01:57.799 INFO:teuthology.orchestra.run.smithi145.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-04-22T07:01:57.811 INFO:teuthology.orchestra.run.smithi092.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-04-22T07:01:57.841 INFO:teuthology.orchestra.run.smithi145.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 904 kB/s | 159 kB 00:00 2024-04-22T07:01:57.844 INFO:teuthology.orchestra.run.smithi092.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 262 kB/s | 31 kB 00:00 2024-04-22T07:01:57.866 INFO:teuthology.orchestra.run.smithi145.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 763 kB/s | 51 kB 00:00 2024-04-22T07:01:57.891 INFO:teuthology.orchestra.run.smithi145.stdout:(90/150): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-04-22T07:01:57.911 INFO:teuthology.orchestra.run.smithi092.stdout:(138/150): python3-kubernetes-26.1.0-2.el9.noar 708 kB/s | 1.0 MB 00:01 2024-04-22T07:01:57.916 INFO:teuthology.orchestra.run.smithi145.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 54 kB 00:00 2024-04-22T07:01:57.967 INFO:teuthology.orchestra.run.smithi145.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-04-22T07:01:57.970 INFO:teuthology.orchestra.run.smithi092.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 472 kB/s | 59 kB 00:00 2024-04-22T07:01:57.992 INFO:teuthology.orchestra.run.smithi145.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 586 kB/s | 44 kB 00:00 2024-04-22T07:01:58.029 INFO:teuthology.orchestra.run.smithi092.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-04-22T07:01:58.042 INFO:teuthology.orchestra.run.smithi145.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-04-22T07:01:58.067 INFO:teuthology.orchestra.run.smithi145.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-04-22T07:01:58.087 INFO:teuthology.orchestra.run.smithi092.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 437 kB/s | 51 kB 00:00 2024-04-22T07:01:58.112 INFO:teuthology.orchestra.run.smithi092.stdout:(142/150): python3-routes-2.5.1-5.el9.noarch.rp 626 kB/s | 188 kB 00:00 2024-04-22T07:01:58.118 INFO:teuthology.orchestra.run.smithi145.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-04-22T07:01:58.168 INFO:teuthology.orchestra.run.smithi145.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-22T07:01:58.205 INFO:teuthology.orchestra.run.smithi092.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 766 kB/s | 90 kB 00:00 2024-04-22T07:01:58.235 INFO:teuthology.orchestra.run.smithi145.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-22T07:01:58.302 INFO:teuthology.orchestra.run.smithi145.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 956 kB/s | 64 kB 00:00 2024-04-22T07:01:58.330 INFO:teuthology.orchestra.run.smithi092.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-04-22T07:01:58.355 INFO:teuthology.orchestra.run.smithi092.stdout:(145/150): python3-webob-1.8.7-6.el9.noarch.rpm 704 kB/s | 230 kB 00:00 2024-04-22T07:01:58.369 INFO:teuthology.orchestra.run.smithi145.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 672 kB/s | 45 kB 00:00 2024-04-22T07:01:58.547 INFO:teuthology.orchestra.run.smithi092.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 884 kB/s | 192 kB 00:00 2024-04-22T07:01:58.561 INFO:teuthology.orchestra.run.smithi145.stdout:(101/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-22T07:01:58.586 INFO:teuthology.orchestra.run.smithi145.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 103 kB/s | 22 kB 00:00 2024-04-22T07:01:58.588 INFO:teuthology.orchestra.run.smithi092.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 897 kB/s | 427 kB 00:00 2024-04-22T07:01:58.711 INFO:teuthology.orchestra.run.smithi145.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-04-22T07:01:58.981 INFO:teuthology.orchestra.run.smithi092.stdout:(148/150): librados2-19.0.0-3156.g86f7587a.el9. 7.8 MB/s | 3.4 MB 00:00 2024-04-22T07:01:59.089 INFO:teuthology.orchestra.run.smithi092.stdout:(149/150): librbd1-19.0.0-3156.g86f7587a.el9.x8 6.3 MB/s | 3.1 MB 00:00 2024-04-22T07:01:59.170 INFO:teuthology.orchestra.run.smithi145.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 939 kB/s | 548 kB 00:00 2024-04-22T07:01:59.221 INFO:teuthology.orchestra.run.smithi145.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 607 kB/s | 309 kB 00:00 2024-04-22T07:01:59.446 INFO:teuthology.orchestra.run.smithi145.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 71 kB/s | 19 kB 00:00 2024-04-22T07:01:59.721 INFO:teuthology.orchestra.run.smithi145.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 94 kB/s | 26 kB 00:00 2024-04-22T07:02:00.063 INFO:teuthology.orchestra.run.smithi145.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-22T07:02:00.206 INFO:teuthology.orchestra.run.smithi092.stdout:(150/150): thrift-0.15.0-2.el9.x86_64.rpm 875 kB/s | 1.6 MB 00:01 2024-04-22T07:02:00.208 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-22T07:02:00.209 INFO:teuthology.orchestra.run.smithi092.stdout:Total 4.7 MB/s | 218 MB 00:46 2024-04-22T07:02:00.230 INFO:teuthology.orchestra.run.smithi145.stdout:(109/150): uuid-1.6.2-55.el9.x86_64.rpm 27 kB/s | 58 kB 00:02 2024-04-22T07:02:00.405 INFO:teuthology.orchestra.run.smithi145.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 197 kB/s | 67 kB 00:00 2024-04-22T07:02:00.589 INFO:teuthology.orchestra.run.smithi145.stdout:(111/150): libarrow-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-04-22T07:02:00.656 INFO:teuthology.orchestra.run.smithi145.stdout:(112/150): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-04-22T07:02:01.006 INFO:teuthology.orchestra.run.smithi145.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 124 kB/s | 43 kB 00:00 2024-04-22T07:02:01.073 INFO:teuthology.orchestra.run.smithi145.stdout:(114/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-22T07:02:01.198 INFO:teuthology.orchestra.run.smithi145.stdout:(115/150): python3-asyncssh-2.13.2-5.el9.noarch 900 kB/s | 548 kB 00:00 2024-04-22T07:02:01.263 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-22T07:02:01.273 INFO:teuthology.orchestra.run.smithi145.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 120 kB/s | 32 kB 00:00 2024-04-22T07:02:01.332 INFO:teuthology.orchestra.run.smithi145.stdout:(117/150): python3-certifi-2021.10.8-2.el9.noar 58 kB/s | 15 kB 00:00 2024-04-22T07:02:01.387 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-22T07:02:01.387 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-22T07:02:01.624 INFO:teuthology.orchestra.run.smithi145.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 404 kB/s | 172 kB 00:00 2024-04-22T07:02:01.782 INFO:teuthology.orchestra.run.smithi145.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2024-04-22T07:02:01.832 INFO:teuthology.orchestra.run.smithi145.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 606 kB/s | 303 kB 00:00 2024-04-22T07:02:01.883 INFO:teuthology.orchestra.run.smithi145.stdout:(121/150): python3-colorama-0.4.4-12.el9.noarch 136 kB/s | 35 kB 00:00 2024-04-22T07:02:02.283 INFO:teuthology.orchestra.run.smithi145.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 405 kB/s | 203 kB 00:00 2024-04-22T07:02:02.316 INFO:teuthology.orchestra.run.smithi145.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 333 kB/s | 144 kB 00:00 2024-04-22T07:02:02.475 INFO:teuthology.orchestra.run.smithi145.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-04-22T07:02:02.584 INFO:teuthology.orchestra.run.smithi145.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 66 kB/s | 18 kB 00:00 2024-04-22T07:02:02.751 INFO:teuthology.orchestra.run.smithi145.stdout:(126/150): python3-grpcio-1.46.7-10.el9.x86_64. 2.2 MB/s | 2.0 MB 00:00 2024-04-22T07:02:02.776 INFO:teuthology.orchestra.run.smithi145.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 77 kB/s | 23 kB 00:00 2024-04-22T07:02:02.843 INFO:teuthology.orchestra.run.smithi145.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 75 kB/s | 19 kB 00:00 2024-04-22T07:02:03.085 INFO:teuthology.orchestra.run.smithi145.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 59 kB/s | 20 kB 00:00 2024-04-22T07:02:03.114 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-22T07:02:03.114 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-22T07:02:03.185 INFO:teuthology.orchestra.run.smithi145.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-04-22T07:02:03.452 INFO:teuthology.orchestra.run.smithi145.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 215 kB/s | 79 kB 00:00 2024-04-22T07:02:03.536 INFO:teuthology.orchestra.run.smithi145.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 1.3 MB/s | 1.0 MB 00:00 2024-04-22T07:02:03.561 INFO:teuthology.orchestra.run.smithi145.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 154 kB/s | 58 kB 00:00 2024-04-22T07:02:03.803 INFO:teuthology.orchestra.run.smithi145.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-04-22T07:02:03.953 INFO:teuthology.orchestra.run.smithi145.stdout:(135/150): python3-pecan-1.4.2-3.el9.noarch.rpm 544 kB/s | 272 kB 00:00 2024-04-22T07:02:03.987 INFO:teuthology.orchestra.run.smithi145.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 211 kB/s | 90 kB 00:00 2024-04-22T07:02:04.070 INFO:teuthology.orchestra.run.smithi145.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 105 kB/s | 28 kB 00:00 2024-04-22T07:02:04.212 INFO:teuthology.orchestra.run.smithi145.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 119 kB/s | 31 kB 00:00 2024-04-22T07:02:04.421 INFO:teuthology.orchestra.run.smithi145.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 169 kB/s | 59 kB 00:00 2024-04-22T07:02:04.446 INFO:teuthology.orchestra.run.smithi145.stdout:(140/150): python3-routes-2.5.1-5.el9.noarch.rp 411 kB/s | 188 kB 00:00 2024-04-22T07:02:04.480 INFO:teuthology.orchestra.run.smithi145.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-04-22T07:02:04.763 INFO:teuthology.orchestra.run.smithi145.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 149 kB/s | 51 kB 00:00 2024-04-22T07:02:04.822 INFO:teuthology.orchestra.run.smithi145.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 262 kB/s | 90 kB 00:00 2024-04-22T07:02:04.955 INFO:teuthology.orchestra.run.smithi145.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 451 kB/s | 230 kB 00:00 2024-04-22T07:02:05.089 INFO:teuthology.orchestra.run.smithi145.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-04-22T07:02:05.364 INFO:teuthology.orchestra.run.smithi145.stdout:(146/150): python3-werkzeug-2.0.3-3.el9.1.noarc 711 kB/s | 427 kB 00:00 2024-04-22T07:02:05.389 INFO:teuthology.orchestra.run.smithi145.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 442 kB/s | 192 kB 00:00 2024-04-22T07:02:05.715 INFO:teuthology.orchestra.run.smithi145.stdout:(148/150): librbd1-19.0.0-3156.g86f7587a.el9.x8 9.7 MB/s | 3.1 MB 00:00 2024-04-22T07:02:05.848 INFO:teuthology.orchestra.run.smithi145.stdout:(149/150): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-04-22T07:02:06.440 INFO:teuthology.orchestra.run.smithi145.stdout:(150/150): librados2-19.0.0-3156.g86f7587a.el9. 3.2 MB/s | 3.4 MB 00:01 2024-04-22T07:02:06.443 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-04-22T07:02:06.443 INFO:teuthology.orchestra.run.smithi145.stdout:Total 6.7 MB/s | 218 MB 00:32 2024-04-22T07:02:07.474 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-04-22T07:02:07.599 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-04-22T07:02:07.599 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-04-22T07:02:08.894 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-22T07:02:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T07:02:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T07:02:09.050 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T07:02:09.313 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-04-22T07:02:09.313 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-04-22T07:02:09.379 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T07:02:09.500 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:02:09.622 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:02:09.659 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:02:09.743 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:02:09.784 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:02:09.822 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T07:02:09.847 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T07:02:10.006 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T07:02:10.053 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T07:02:10.081 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:02:10.184 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:02:10.207 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:02:10.261 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:02:10.283 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:02:10.307 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:02:10.491 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:02:10.524 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:02:10.632 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T07:02:10.730 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T07:02:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T07:02:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T07:02:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T07:02:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T07:02:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T07:02:11.000 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T07:02:11.036 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T07:02:11.106 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T07:02:11.159 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T07:02:11.188 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T07:02:11.224 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T07:02:11.301 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T07:02:11.334 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 30/152 2024-04-22T07:02:11.382 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 31/152 2024-04-22T07:02:11.406 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 32/152 2024-04-22T07:02:11.479 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T07:02:11.530 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T07:02:11.606 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T07:02:11.743 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T07:02:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T07:02:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T07:02:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T07:02:12.465 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T07:02:12.507 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T07:02:12.574 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 42/152 2024-04-22T07:02:12.604 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T07:02:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T07:02:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T07:02:12.779 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T07:02:12.815 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T07:02:12.863 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T07:02:12.898 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T07:02:12.942 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T07:02:12.984 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T07:02:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T07:02:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T07:02:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T07:02:13.252 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T07:02:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T07:02:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T07:02:13.425 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T07:02:13.450 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T07:02:13.475 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T07:02:13.551 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T07:02:13.688 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T07:02:13.724 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T07:02:13.752 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T07:02:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T07:02:13.831 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T07:02:13.868 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T07:02:13.908 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T07:02:13.937 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T07:02:13.963 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T07:02:13.997 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T07:02:14.038 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T07:02:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T07:02:14.163 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T07:02:14.237 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T07:02:14.268 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T07:02:14.315 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T07:02:14.416 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T07:02:14.478 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T07:02:15.046 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-04-22T07:02:15.125 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T07:02:15.159 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T07:02:15.201 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T07:02:15.231 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T07:02:15.283 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T07:02:15.317 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T07:02:15.420 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T07:02:15.737 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T07:02:15.737 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:02:15.737 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T07:02:15.737 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T07:02:15.737 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T07:02:15.766 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:02:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T07:02:15.803 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:02:15.807 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T07:02:15.885 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T07:02:15.893 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:02:15.937 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:02:15.965 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T07:02:15.991 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T07:02:16.142 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T07:02:16.196 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T07:02:16.224 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:02:16.328 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:02:16.350 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:02:16.413 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:02:16.427 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:02:16.451 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:02:16.559 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T07:02:16.652 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:02:16.684 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:02:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T07:02:16.784 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T07:02:16.880 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T07:02:16.908 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T07:02:16.951 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T07:02:17.017 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T07:02:17.050 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T07:02:17.102 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T07:02:17.135 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T07:02:17.163 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T07:02:17.233 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T07:02:17.285 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T07:02:17.315 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T07:02:17.351 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T07:02:17.428 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T07:02:17.462 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 30/152 2024-04-22T07:02:17.496 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 31/152 2024-04-22T07:02:17.525 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 32/152 2024-04-22T07:02:17.596 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T07:02:17.640 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T07:02:17.715 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T07:02:17.847 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T07:02:17.901 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T07:02:17.959 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T07:02:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T07:02:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T07:02:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T07:02:18.418 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T07:02:18.447 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T07:02:18.460 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:02:18.532 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T07:02:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:02:18.559 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T07:02:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 98/152 2024-04-22T07:02:18.602 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T07:02:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T07:02:18.640 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 42/152 2024-04-22T07:02:18.673 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T07:02:18.728 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T07:02:18.802 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T07:02:18.832 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T07:02:18.869 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T07:02:18.915 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T07:02:18.951 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T07:02:18.995 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T07:02:19.037 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T07:02:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T07:02:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:02:19.175 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T07:02:19.192 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:02:19.222 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 102/152 2024-04-22T07:02:19.225 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T07:02:19.255 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T07:02:19.297 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T07:02:19.392 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T07:02:19.430 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T07:02:19.470 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T07:02:19.496 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T07:02:19.521 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T07:02:19.596 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T07:02:19.732 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T07:02:19.769 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T07:02:19.798 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T07:02:19.832 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T07:02:19.876 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T07:02:19.913 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T07:02:19.945 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T07:02:19.975 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T07:02:20.001 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T07:02:20.034 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T07:02:20.076 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T07:02:20.110 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T07:02:20.198 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T07:02:20.266 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T07:02:20.297 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T07:02:20.344 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T07:02:20.443 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T07:02:20.498 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T07:02:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T07:02:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T07:02:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T07:02:21.416 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T07:02:21.479 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T07:02:21.516 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:02:21.552 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:02:21.566 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T07:02:21.597 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T07:02:21.775 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T07:02:21.804 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T07:02:21.890 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T07:02:22.092 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:02:22.130 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:02:22.212 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:02:22.212 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T07:02:22.212 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T07:02:22.212 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:22.239 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:02:22.543 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T07:02:22.645 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T07:02:23.898 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T07:02:23.966 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T07:02:23.994 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T07:02:24.027 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T07:02:24.338 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T07:02:24.381 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:02:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:02:24.503 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 98/152 2024-04-22T07:02:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T07:02:25.021 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T07:02:25.059 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:02:25.130 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:02:25.161 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 102/152 2024-04-22T07:02:27.478 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:02:27.515 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:02:28.055 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:02:28.094 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:02:28.183 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:02:28.183 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T07:02:28.183 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T07:02:28.183 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:02:28.212 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:02:33.591 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:02:33.592 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-04-22T07:02:33.592 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-04-22T07:02:33.592 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-04-22T07:02:33.592 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-04-22T07:02:33.592 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-04-22T07:02:33.592 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-04-22T07:02:33.592 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-04-22T07:02:33.592 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:33.784 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:02:33.799 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:02:33.800 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:33.800 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T07:02:33.800 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:35.295 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:02:35.321 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:02:35.321 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:35.321 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T07:02:35.321 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:02:35.321 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:02:35.321 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:35.545 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:02:35.568 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:02:35.568 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:35.568 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T07:02:35.568 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:02:35.568 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:02:35.568 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:36.021 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:02:36.044 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:02:36.044 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:36.044 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T07:02:36.044 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:02:36.044 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:02:36.044 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T07:02:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T07:02:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T07:02:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T07:02:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T07:02:36.576 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T07:02:36.664 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T07:02:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T07:02:36.839 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T07:02:36.870 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T07:02:36.898 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T07:02:37.383 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:02:37.413 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:02:37.446 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 122/152 2024-04-22T07:02:37.507 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 123/152 2024-04-22T07:02:37.545 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:02:38.590 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:02:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:02:39.589 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:02:39.621 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:02:39.658 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:02:39.658 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /sys 2024-04-22T07:02:39.658 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /proc 2024-04-22T07:02:39.658 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /mnt 2024-04-22T07:02:39.658 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /var/tmp 2024-04-22T07:02:39.658 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /home 2024-04-22T07:02:39.658 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /root 2024-04-22T07:02:39.658 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /tmp 2024-04-22T07:02:39.659 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:02:39.725 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:02:39.849 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:02:39.865 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:02:39.865 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:39.865 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T07:02:39.865 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:02:39.887 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 127/152 2024-04-22T07:02:39.924 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:02:39.947 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:02:39.947 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:39.947 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T07:02:39.947 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:02:39.948 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:02:39.948 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:02:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:02:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 130/152 2024-04-22T07:02:40.148 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 131/152 2024-04-22T07:02:41.078 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T07:02:41.110 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:02:41.134 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:02:41.134 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:41.134 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T07:02:41.134 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:02:41.134 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:02:41.135 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:41.177 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:02:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:02:41.200 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:41.200 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T07:02:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:02:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:02:41.267 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:41.267 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T07:02:41.267 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:02:41.268 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:02:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:41.352 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:02:41.377 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:02:41.377 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:41.377 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T07:02:41.377 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:02:41.377 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:02:41.377 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:02:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:02:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:02:41.602 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:41.602 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T07:02:41.602 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:02:41.602 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:02:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:41.603 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:02:41.625 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:02:41.625 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:41.625 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T07:02:41.625 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:02:41.625 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:02:41.625 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:02:42.070 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:02:42.092 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:02:42.092 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:42.093 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T07:02:42.093 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:02:42.093 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:02:42.093 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:02:42.191 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T07:02:42.248 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T07:02:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T07:02:42.486 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T07:02:42.537 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T07:02:42.614 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T07:02:42.710 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T07:02:42.840 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T07:02:42.878 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T07:02:42.911 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T07:02:42.938 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T07:02:43.367 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:02:43.395 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:02:43.428 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 122/152 2024-04-22T07:02:43.488 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 123/152 2024-04-22T07:02:43.518 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:02:44.558 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:02:44.595 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:02:45.563 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:02:45.596 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:02:45.696 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:02:45.850 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 127/152 2024-04-22T07:02:45.881 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:02:45.904 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:02:45.904 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:45.904 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T07:02:45.904 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:02:45.904 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:02:45.904 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:02:45.940 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:02:45.951 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:02:46.048 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 130/152 2024-04-22T07:02:46.105 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 131/152 2024-04-22T07:02:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 137/152 2024-04-22T07:02:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 138/152 2024-04-22T07:02:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 139/152 2024-04-22T07:02:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T07:02:46.365 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T07:02:46.554 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 142/152 2024-04-22T07:02:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T07:02:46.633 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 144/152 2024-04-22T07:02:47.029 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T07:02:47.059 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:02:47.082 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:02:47.082 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:47.082 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T07:02:47.083 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:02:47.083 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:02:47.083 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:02:47.126 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:02:47.148 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:02:47.148 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:47.148 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T07:02:47.148 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:02:47.194 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:02:47.216 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:02:47.216 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:47.216 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T07:02:47.216 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:02:47.217 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:02:47.217 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:02:47.234 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:02:47.271 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:02:47.291 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:02:47.511 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:02:47.534 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:02:47.534 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:47.534 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T07:02:47.535 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:02:47.535 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:02:47.535 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:02:48.071 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 146/152 2024-04-22T07:02:48.159 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T07:02:48.200 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T07:02:48.281 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T07:02:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T07:02:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:02:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:02:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:02:52.114 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 137/152 2024-04-22T07:02:52.155 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 138/152 2024-04-22T07:02:52.196 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 139/152 2024-04-22T07:02:52.228 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T07:02:52.255 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T07:02:52.436 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 142/152 2024-04-22T07:02:52.485 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T07:02:52.515 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 144/152 2024-04-22T07:02:53.143 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:02:53.187 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:02:53.207 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:02:53.983 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 146/152 2024-04-22T07:02:54.075 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T07:02:54.116 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T07:02:54.197 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T07:02:54.229 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T07:02:54.229 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:02:54.322 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:02:54.322 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:02:54.375 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:02:54.375 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 1/152 2024-04-22T07:02:54.375 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2/152 2024-04-22T07:02:54.375 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 3/152 2024-04-22T07:02:54.375 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 4/152 2024-04-22T07:02:54.375 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 5/152 2024-04-22T07:02:54.375 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:02:54.375 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:02:54.375 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 8/152 2024-04-22T07:02:54.375 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 9/152 2024-04-22T07:02:54.375 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 10/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 11/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 13/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 14/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 15/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 16/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 17/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 18/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 19/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 20/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 21/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 22/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 23/152 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 24/152 2024-04-22T07:02:54.377 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 25/152 2024-04-22T07:02:54.377 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 26/152 2024-04-22T07:02:54.377 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 27/152 2024-04-22T07:02:54.377 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 28/152 2024-04-22T07:02:54.377 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 29/152 2024-04-22T07:02:54.388 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 30/152 2024-04-22T07:02:54.388 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 31/152 2024-04-22T07:02:54.388 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 32/152 2024-04-22T07:02:54.388 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 33/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 34/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 35/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 36/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 37/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 38/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 39/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T07:02:54.389 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T07:02:54.390 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T07:02:54.391 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T07:02:54.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T07:02:54.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T07:02:54.394 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T07:02:54.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T07:02:54.396 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T07:02:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T07:02:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T07:02:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T07:02:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T07:02:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T07:02:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 149/152 2024-04-22T07:02:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T07:02:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 151/152 2024-04-22T07:02:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:02:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:56.425 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-04-22T07:02:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.425 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-22T07:02:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T07:02:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T07:02:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T07:02:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T07:02:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T07:02:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T07:02:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T07:02:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T07:02:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T07:02:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T07:02:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T07:02:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T07:02:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T07:02:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T07:02:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T07:02:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T07:02:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T07:02:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T07:02:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T07:02:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T07:02:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T07:02:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T07:02:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T07:02:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T07:02:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T07:02:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T07:02:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T07:02:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:02:56.436 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-22T07:02:56.787 DEBUG:teuthology.parallel:result is None 2024-04-22T07:03:00.644 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:03:00.644 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 1/152 2024-04-22T07:03:00.644 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2/152 2024-04-22T07:03:00.644 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 3/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 4/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 5/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 8/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 9/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 10/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 11/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 13/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 14/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 15/152 2024-04-22T07:03:00.645 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 16/152 2024-04-22T07:03:00.646 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 17/152 2024-04-22T07:03:00.646 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 18/152 2024-04-22T07:03:00.646 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 19/152 2024-04-22T07:03:00.646 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 20/152 2024-04-22T07:03:00.646 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 21/152 2024-04-22T07:03:00.646 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 22/152 2024-04-22T07:03:00.646 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 23/152 2024-04-22T07:03:00.646 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 24/152 2024-04-22T07:03:00.646 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 25/152 2024-04-22T07:03:00.646 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 26/152 2024-04-22T07:03:00.646 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 27/152 2024-04-22T07:03:00.646 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 28/152 2024-04-22T07:03:00.658 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 29/152 2024-04-22T07:03:00.658 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 30/152 2024-04-22T07:03:00.658 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 31/152 2024-04-22T07:03:00.658 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 32/152 2024-04-22T07:03:00.658 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 33/152 2024-04-22T07:03:00.658 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 34/152 2024-04-22T07:03:00.658 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 35/152 2024-04-22T07:03:00.658 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 36/152 2024-04-22T07:03:00.658 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 37/152 2024-04-22T07:03:00.658 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 38/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 39/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T07:03:00.659 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T07:03:00.660 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T07:03:00.661 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T07:03:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T07:03:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T07:03:00.665 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T07:03:00.666 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T07:03:00.667 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T07:03:00.667 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 149/152 2024-04-22T07:03:00.667 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T07:03:00.667 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 151/152 2024-04-22T07:03:02.641 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:03:02.641 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:02.641 INFO:teuthology.orchestra.run.smithi145.stdout:Upgraded: 2024-04-22T07:03:02.641 INFO:teuthology.orchestra.run.smithi145.stdout: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.641 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.641 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-04-22T07:03:02.641 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T07:03:02.641 INFO:teuthology.orchestra.run.smithi145.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T07:03:02.642 INFO:teuthology.orchestra.run.smithi145.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T07:03:02.642 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T07:03:02.642 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.642 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.642 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.642 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.642 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:02.642 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.642 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.642 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.642 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:02.642 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.643 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:02.644 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T07:03:02.644 INFO:teuthology.orchestra.run.smithi145.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T07:03:02.644 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T07:03:02.644 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T07:03:02.644 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T07:03:02.644 INFO:teuthology.orchestra.run.smithi145.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T07:03:02.644 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T07:03:02.644 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T07:03:02.644 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T07:03:02.644 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T07:03:02.644 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T07:03:02.644 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T07:03:02.645 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T07:03:02.646 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T07:03:02.647 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T07:03:02.648 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T07:03:02.649 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T07:03:02.650 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T07:03:02.651 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T07:03:02.652 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T07:03:02.652 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T07:03:02.652 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.652 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.652 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:02.652 INFO:teuthology.orchestra.run.smithi145.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T07:03:02.652 INFO:teuthology.orchestra.run.smithi145.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T07:03:02.652 INFO:teuthology.orchestra.run.smithi145.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T07:03:02.652 INFO:teuthology.orchestra.run.smithi145.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T07:03:02.652 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T07:03:02.652 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T07:03:02.653 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T07:03:02.653 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:02.653 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-04-22T07:03:03.044 DEBUG:teuthology.parallel:result is None 2024-04-22T07:03:03.045 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:03:03.196 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T07:03:03.221 INFO:teuthology.orchestra.run.smithi092.stdout:19.0.0-3156.g86f7587a.el9 2024-04-22T07:03:03.222 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.g86f7587a.el9 2024-04-22T07:03:03.222 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.g86f7587a is installed. 2024-04-22T07:03:03.224 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:03:03.372 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T07:03:03.393 INFO:teuthology.orchestra.run.smithi145.stdout:19.0.0-3156.g86f7587a.el9 2024-04-22T07:03:03.393 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.g86f7587a.el9 2024-04-22T07:03:03.393 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.g86f7587a is installed. 2024-04-22T07:03:03.395 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-22T07:03:03.395 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:03.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T07:03:03.426 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:03.426 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T07:03:03.459 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-22T07:03:03.459 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:03.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T07:03:03.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T07:03:03.561 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:03.561 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T07:03:03.585 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T07:03:03.650 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-22T07:03:03.650 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:03.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T07:03:03.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T07:03:03.744 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:03.744 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T07:03:03.768 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T07:03:03.832 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-22T07:03:03.833 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:03.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T07:03:03.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T07:03:03.924 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:03.924 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T07:03:03.947 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T07:03:04.014 INFO:teuthology.run_tasks:Running task ceph... 2024-04-22T07:03:04.102 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-22T07:03:04.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2024-04-22T07:03:04.105 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /var/log/ceph 2024-04-22T07:03:04.129 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-22T07:03:04.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-22T07:03:04.171 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-22T07:03:04.194 INFO:tasks.ceph:Creating extra log directories... 2024-04-22T07:03:04.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-22T07:03:04.236 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-22T07:03:04.267 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-22T07:03:04.267 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-22T07:03:04.267 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-22T07:03:04.268 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:04.268 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-22T07:03:04.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-22T07:03:04.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-22T07:03:04.463 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:04.463 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-22T07:03:04.478 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-22T07:03:04.804 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-22T07:03:04.835 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-22T07:03:04.835 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-22T07:03:04.835 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159', 'branch': 'wip-lusov-quiescer', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/fs/snaps}}', 'email': None, 'first_in_suite': False, 'job_id': '7668159', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-quiescer'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-quiescer', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 4362, 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'sleep_before_teardown': 0, 'subset': '222/111111', 'suite': 'fs', 'suite_branch': 'wip-lusov-quiescer', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJVl31wtivxBEbjpTOCjOi5KvLj1vB5LcpWY56N+WMhEvOx32lnLtLoh1Bq8+LgeR9qUCXErSnly6rZdEGibL6Y=', 'smithi145.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDTO262E0ybwmBW+GUof6kOF2K0vgwR1F/9MxdtBjNRvP52H+6lALEgMCoJarHGCC43MXkctFfZm4ttjS+xK7XQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'quiescer': None}, {'workunit': {'clients': {'all': ['fs/snaps']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-22_05:52:53', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-22T07:03:04.836 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-22T07:03:04.839 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-22T07:03:04.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2024-04-22T07:03:04.894 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m0777 -- /var/run/ceph 2024-04-22T07:03:04.948 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:04.948 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T07:03:04.971 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T07:03:04.971 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-04-22T07:03:05.026 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T07:03:05.027 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:05.027 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-22T07:03:05.027 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:05.027 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:05.027 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-22 07:02:48.860687190 +0000 2024-04-22T07:03:05.027 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-22 07:00:20.821342771 +0000 2024-04-22T07:03:05.027 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-22 07:00:20.821342771 +0000 2024-04-22T07:03:05.027 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-22 07:00:20.821342771 +0000 2024-04-22T07:03:05.027 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T07:03:05.088 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-22T07:03:05.088 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-22T07:03:05.089 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000200555 s, 2.6 MB/s 2024-04-22T07:03:05.089 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T07:03:05.147 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-04-22T07:03:05.203 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T07:03:05.203 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:05.203 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-22T07:03:05.203 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:05.203 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:05.203 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-22 07:02:48.860687190 +0000 2024-04-22T07:03:05.203 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-22 07:00:21.153333413 +0000 2024-04-22T07:03:05.203 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-22 07:00:21.153333413 +0000 2024-04-22T07:03:05.203 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-22 07:00:21.153333413 +0000 2024-04-22T07:03:05.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T07:03:05.264 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-22T07:03:05.264 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-22T07:03:05.264 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000160987 s, 3.2 MB/s 2024-04-22T07:03:05.266 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T07:03:05.321 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-04-22T07:03:05.375 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T07:03:05.375 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:05.375 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-22T07:03:05.375 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:05.375 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:05.375 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-22 07:02:48.860687190 +0000 2024-04-22T07:03:05.375 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-22 07:00:21.486324025 +0000 2024-04-22T07:03:05.375 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-22 07:00:21.486324025 +0000 2024-04-22T07:03:05.375 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-22 07:00:21.486324025 +0000 2024-04-22T07:03:05.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T07:03:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-22T07:03:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-22T07:03:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000162859 s, 3.1 MB/s 2024-04-22T07:03:05.437 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T07:03:05.493 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-04-22T07:03:05.548 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T07:03:05.548 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:05.548 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-22T07:03:05.548 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:05.548 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:05.549 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-22 07:02:48.861687161 +0000 2024-04-22T07:03:05.549 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-22 07:00:21.828314385 +0000 2024-04-22T07:03:05.549 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-22 07:00:21.828314385 +0000 2024-04-22T07:03:05.549 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-22 07:00:21.828314385 +0000 2024-04-22T07:03:05.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T07:03:05.610 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-22T07:03:05.610 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-22T07:03:05.610 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000177816 s, 2.9 MB/s 2024-04-22T07:03:05.611 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T07:03:05.667 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:05.667 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:05.667 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T07:03:05.683 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T07:03:05.683 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_1 2024-04-22T07:03:05.739 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T07:03:05.739 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:05.739 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-22T07:03:05.739 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:05.739 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:05.740 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-04-22 07:02:54.813313417 +0000 2024-04-22T07:03:05.740 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-04-22 07:00:21.399276508 +0000 2024-04-22T07:03:05.740 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-04-22 07:00:21.399276508 +0000 2024-04-22T07:03:05.740 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-04-22 07:00:21.399276508 +0000 2024-04-22T07:03:05.740 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T07:03:05.802 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-04-22T07:03:05.803 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-04-22T07:03:05.803 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000322759 s, 1.6 MB/s 2024-04-22T07:03:05.804 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T07:03:05.862 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_2 2024-04-22T07:03:05.917 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T07:03:05.918 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:05.918 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-22T07:03:05.918 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:05.919 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:05.919 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-04-22 07:02:54.813313417 +0000 2024-04-22T07:03:05.919 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-04-22 07:00:21.753268902 +0000 2024-04-22T07:03:05.919 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-04-22 07:00:21.753268902 +0000 2024-04-22T07:03:05.919 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-04-22 07:00:21.753268902 +0000 2024-04-22T07:03:05.920 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T07:03:05.980 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-04-22T07:03:05.980 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-04-22T07:03:05.980 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000191482 s, 2.7 MB/s 2024-04-22T07:03:05.982 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T07:03:06.038 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_3 2024-04-22T07:03:06.093 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T07:03:06.093 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:06.093 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-22T07:03:06.093 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:06.093 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:06.093 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-04-22 07:02:54.813313417 +0000 2024-04-22T07:03:06.093 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-04-22 07:00:22.098261490 +0000 2024-04-22T07:03:06.093 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-04-22 07:00:22.098261490 +0000 2024-04-22T07:03:06.093 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-04-22 07:00:22.098261490 +0000 2024-04-22T07:03:06.094 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T07:03:06.155 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-04-22T07:03:06.155 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-04-22T07:03:06.155 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000225109 s, 2.3 MB/s 2024-04-22T07:03:06.156 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T07:03:06.212 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_4 2024-04-22T07:03:06.267 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T07:03:06.267 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:06.267 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-22T07:03:06.267 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:06.267 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:06.267 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-04-22 07:02:54.814313395 +0000 2024-04-22T07:03:06.267 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-04-22 07:00:22.424254486 +0000 2024-04-22T07:03:06.267 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-04-22 07:00:22.424254486 +0000 2024-04-22T07:03:06.267 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-04-22 07:00:22.424254486 +0000 2024-04-22T07:03:06.268 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T07:03:06.329 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-04-22T07:03:06.329 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-04-22T07:03:06.329 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000173637 s, 2.9 MB/s 2024-04-22T07:03:06.330 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T07:03:06.385 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:06.385 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi145.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-22T07:03:06.385 INFO:tasks.ceph:Generating config... 2024-04-22T07:03:06.387 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-22T07:03:06.387 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-04-22T07:03:06.387 INFO:tasks.ceph:[client] debug client = 20 2024-04-22T07:03:06.387 INFO:tasks.ceph:[client] debug ms = 1 2024-04-22T07:03:06.387 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-22T07:03:06.387 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-22T07:03:06.387 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-22T07:03:06.387 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-22T07:03:06.387 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-22T07:03:06.387 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-04-22T07:03:06.387 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-22T07:03:06.388 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-22T07:03:06.389 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-22T07:03:06.389 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-22T07:03:06.389 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-22T07:03:06.389 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-22T07:03:06.389 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-22T07:03:06.389 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-22T07:03:06.389 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-22T07:03:06.389 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-22T07:03:06.389 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-22T07:03:06.389 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-22T07:03:06.389 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-22T07:03:06.389 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-22T07:03:06.389 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-22T07:03:06.389 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-22T07:03:06.389 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-22T07:03:06.390 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-22T07:03:06.390 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-22T07:03:06.390 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-22T07:03:06.390 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-22T07:03:06.390 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-22T07:03:06.390 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-22T07:03:06.390 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-22T07:03:06.390 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-22T07:03:06.390 INFO:tasks.ceph:Setting up mon.a... 2024-04-22T07:03:06.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-22T07:03:06.440 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.keyring 2024-04-22T07:03:06.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-22T07:03:06.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T07:03:06.545 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.92'), ('mon.b', '172.21.15.145'), ('mon.c', '[v2:172.21.15.145:3301,v1:172.21.15.145:6790]')] 2024-04-22T07:03:06.546 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.92,172.21.15.145,[v2:172.21.15.145:3301,v1:172.21.15.145:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-22T07:03:06.547 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:06.547 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-22T07:03:06.600 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.92 --add b 172.21.15.145 --addv c '[v2:172.21.15.145:3301,v1:172.21.15.145:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:06.685 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:06.685 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: generated fsid 72eb29c5-e441-491c-9309-4fb4ef16ce27 2024-04-22T07:03:06.685 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = quincy 2024-04-22T07:03:06.685 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 0 2024-04-22T07:03:06.685 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 72eb29c5-e441-491c-9309-4fb4ef16ce27 2024-04-22T07:03:06.685 INFO:teuthology.orchestra.run.smithi092.stdout:last_changed 2024-04-22T07:03:06.688250+0000 2024-04-22T07:03:06.685 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-04-22T07:03:06.688250+0000 2024-04-22T07:03:06.685 INFO:teuthology.orchestra.run.smithi092.stdout:min_mon_release 17 (quincy) 2024-04-22T07:03:06.685 INFO:teuthology.orchestra.run.smithi092.stdout:election_strategy: 1 2024-04-22T07:03:06.686 INFO:teuthology.orchestra.run.smithi092.stdout:0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-04-22T07:03:06.686 INFO:teuthology.orchestra.run.smithi092.stdout:1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.b 2024-04-22T07:03:06.686 INFO:teuthology.orchestra.run.smithi092.stdout:2: [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] mon.c 2024-04-22T07:03:06.686 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-22T07:03:06.687 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-22T07:03:06.701 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 72eb29c5-e441-491c-9309-4fb4ef16ce27... 2024-04-22T07:03:06.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-22T07:03:06.744 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-22T07:03:06.781 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-04-22T07:03:06.781 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2024-04-22T07:03:06.781 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-22T07:03:06.781 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2024-04-22T07:03:06.781 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2024-04-22T07:03:06.782 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-22T07:03:06.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.92,172.21.15.145,[v2:172.21.15.145:3301,v1:172.21.15.145:6790]" 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 72eb29c5-e441-491c-9309-4fb4ef16ce27 2024-04-22T07:03:06.784 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2024-04-22T07:03:06.785 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.786 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2024-04-22T07:03:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: osd blocked scrub grace period = 3600 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: osd max markdown count = 1000 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-04-22T07:03:06.788 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: debug client = 20 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.789 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: mon down mkfs grace = 300 2024-04-22T07:03:06.790 INFO:teuthology.orchestra.run.smithi092.stdout: mon op complaint time = 120 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout: client mount timeout = 600 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout: client_shutdown_timeout = 120 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout: debug client = 20 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2024-04-22T07:03:06.791 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.e] 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.f] 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout:[mds] 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds = 20 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds balancer = 20 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout: mds bal fragment size max = 10000 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout: mds bal merge size = 5 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout: mds bal split bits = 3 2024-04-22T07:03:06.792 INFO:teuthology.orchestra.run.smithi092.stdout: mds bal split size = 100 2024-04-22T07:03:06.793 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug frag = True 2024-04-22T07:03:06.793 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug scatterstat = True 2024-04-22T07:03:06.793 INFO:teuthology.orchestra.run.smithi092.stdout: mds oft prefetch dirfrags = True 2024-04-22T07:03:06.793 INFO:teuthology.orchestra.run.smithi092.stdout: mds op complaint time = 180 2024-04-22T07:03:06.793 INFO:teuthology.orchestra.run.smithi092.stdout: mds verify scatter = True 2024-04-22T07:03:06.793 INFO:teuthology.orchestra.run.smithi092.stdout: mds_dir_prefetch = True 2024-04-22T07:03:06.793 INFO:teuthology.orchestra.run.smithi092.stdout: mds_sleep_rank_change = 5000000.0 2024-04-22T07:03:06.793 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2024-04-22T07:03:06.793 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2024-04-22T07:03:06.793 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2024-04-22T07:03:06.794 INFO:teuthology.orchestra.run.smithi145.stdout:[global] 2024-04-22T07:03:06.794 INFO:teuthology.orchestra.run.smithi145.stdout: chdir = "" 2024-04-22T07:03:06.794 INFO:teuthology.orchestra.run.smithi145.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-22T07:03:06.794 INFO:teuthology.orchestra.run.smithi145.stdout: auth supported = cephx 2024-04-22T07:03:06.794 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.794 INFO:teuthology.orchestra.run.smithi145.stdout: filestore xattr use omap = true 2024-04-22T07:03:06.794 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.794 INFO:teuthology.orchestra.run.smithi145.stdout: mon clock drift allowed = 1.000 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: osd crush chooseleaf type = 0 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: auth debug = true 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on old message = true 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on bug = true 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: mon pg warn max object skew = 0 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: # disable pg_autoscaler by default for new pools 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.795 INFO:teuthology.orchestra.run.smithi145.stdout: osd pool default size = 2 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd allow primary affinity = true 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd allow pg remap = true 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on legacy crush tunables = false 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on crush straw calc version zero = false 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on no sortbitwise = false 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on osd down out interval zero = false 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on too few osds = false 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: mon_allow_pool_size_one = true 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.796 INFO:teuthology.orchestra.run.smithi145.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: osd default data pool replay window = 5 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: mon allow pool delete = true 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: mon cluster log file level = debug 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: debug asserts on shutdown = true 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: mon health detail to clog = false 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: mon host = "172.21.15.92,172.21.15.145,[v2:172.21.15.145:3301,v1:172.21.15.145:6790]" 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: fsid = 72eb29c5-e441-491c-9309-4fb4ef16ce27 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout:[osd] 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: osd journal size = 100 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.797 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub load threshold = 5.0 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub max interval = 600 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: osd mclock profile = high_recovery_ops 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: osd recover clone overlap = true 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: osd recovery max chunk = 1048576 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug shutdown = true 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug op order = true 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug verify stray on activate = true 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug trim objects = true 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: osd open classes on start = true 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug pg log writeout = true 2024-04-22T07:03:06.798 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: osd deep scrub update digest min age = 30 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: osd map max advance = 10 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: journal zero on create = true 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: filestore ondisk finisher threads = 3 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: filestore apply finisher threads = 3 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: bdev debug aio = true 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug misdirected ops = true 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: bdev async discard = True 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: bdev enable discard = True 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore allocator = bitmap 2024-04-22T07:03:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore block size = 96636764160 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore fsck on mount = True 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: debug bluefs = 1/20 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: debug bluestore = 1/20 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: debug osd = 20 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: debug rocksdb = 4/10 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd backfillfull_ratio = 0.85 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd full ratio = 0.9 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd nearfull ratio = 0.8 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: osd blocked scrub grace period = 3600 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: osd failsafe full ratio = 0.95 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: osd max markdown count = 1000 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: osd objectstore = bluestore 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: osd op complaint time = 180 2024-04-22T07:03:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr] 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: debug mgr = 20 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: debug mon = 20 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: debug auth = 20 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min pgs per osd = 4 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min bytes per osd = 10 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: mgr/telemetry/nag = false 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: debug client = 20 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout:[mon] 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: debug mon = 20 2024-04-22T07:03:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: debug paxos = 20 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: debug auth = 20 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: mon data avail warn = 5 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: mon mgr mkfs grace = 240 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min pgs per osd = 4 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd reporter subtree level = osd 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd prime pg temp = true 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min bytes per osd = 10 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: auth mon ticket ttl = 660 # 11m 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: auth service ticket ttl = 240 # 4m 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: # don't complain about insecure global_id in the test suite 2024-04-22T07:03:06.802 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-22T07:03:06.803 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-22T07:03:06.803 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.803 INFO:teuthology.orchestra.run.smithi145.stdout: # 1m isn't quite enough 2024-04-22T07:03:06.803 INFO:teuthology.orchestra.run.smithi145.stdout: mon_down_mkfs_grace = 2m 2024-04-22T07:03:06.803 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.803 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_filestore_osds = false 2024-04-22T07:03:06.803 INFO:teuthology.orchestra.run.smithi145.stdout: mon down mkfs grace = 300 2024-04-22T07:03:06.803 INFO:teuthology.orchestra.run.smithi145.stdout: mon op complaint time = 120 2024-04-22T07:03:06.803 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-22T07:03:06.803 INFO:teuthology.orchestra.run.smithi145.stdout:[client] 2024-04-22T07:03:06.803 INFO:teuthology.orchestra.run.smithi145.stdout: rgw cache enabled = true 2024-04-22T07:03:06.803 INFO:teuthology.orchestra.run.smithi145.stdout: rgw enable ops log = true 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout: rgw enable usage log = true 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout: client mount timeout = 600 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout: client_shutdown_timeout = 120 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout: debug client = 20 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout: rados mon op timeout = 900 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout: rados osd op timeout = 900 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.a] 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.b] 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.c] 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.a] 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.c] 2024-04-22T07:03:06.804 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.e] 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.b] 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.d] 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.f] 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout:[mds] 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout: debug mds = 20 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout: debug mds balancer = 20 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout: mds bal fragment size max = 10000 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout: mds bal merge size = 5 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout: mds bal split bits = 3 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout: mds bal split size = 100 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout: mds debug frag = True 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout: mds debug scatterstat = True 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout: mds oft prefetch dirfrags = True 2024-04-22T07:03:06.805 INFO:teuthology.orchestra.run.smithi145.stdout: mds op complaint time = 180 2024-04-22T07:03:06.806 INFO:teuthology.orchestra.run.smithi145.stdout: mds verify scatter = True 2024-04-22T07:03:06.806 INFO:teuthology.orchestra.run.smithi145.stdout: mds_dir_prefetch = True 2024-04-22T07:03:06.806 INFO:teuthology.orchestra.run.smithi145.stdout: mds_sleep_rank_change = 5000000.0 2024-04-22T07:03:06.806 INFO:teuthology.orchestra.run.smithi145.stdout: osd op complaint time = 180 2024-04-22T07:03:06.806 INFO:teuthology.orchestra.run.smithi145.stdout: rados mon op timeout = 900 2024-04-22T07:03:06.806 INFO:teuthology.orchestra.run.smithi145.stdout: rados osd op timeout = 900 2024-04-22T07:03:06.807 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-22T07:03:06.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-22T07:03:06.900 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-22T07:03:06.900 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:06.900 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-22T07:03:06.914 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:06.914 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-22T07:03:06.969 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2024-04-22T07:03:06.970 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:06.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-22T07:03:06.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T07:03:07.041 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:07.041 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:07.096 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi145.front.sepia.ceph.com 2024-04-22T07:03:07.096 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:07.096 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-22T07:03:07.097 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T07:03:07.130 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:07.130 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:07.185 INFO:tasks.ceph:Setting up mon nodes... 2024-04-22T07:03:07.185 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-22T07:03:07.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-22T07:03:07.233 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-22T07:03:07.235 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-22T07:03:07.303 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-22T07:03:07.306 INFO:tasks.ceph:Setting up mds nodes... 2024-04-22T07:03:07.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-22T07:03:07.355 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-22T07:03:07.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-22T07:03:07.381 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-22T07:03:07.469 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-22T07:03:07.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-22T07:03:07.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-04-22T07:03:07.583 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-04-22T07:03:07.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-04-22T07:03:07.609 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-22T07:03:07.657 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-22T07:03:07.660 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-22T07:03:07.683 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-22T07:03:07.770 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-22T07:03:07.773 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-22T07:03:07.796 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-04-22T07:03:07.884 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-04-22T07:03:07.886 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-04-22T07:03:07.909 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-22T07:03:07.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-22T07:03:07.949 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-22T07:03:07.961 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-22T07:03:07.961 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi145.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-22T07:03:07.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-22T07:03:08.024 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:08.024 INFO:tasks.ceph:role: osd.0 2024-04-22T07:03:08.025 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-22T07:03:08.025 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-22T07:03:08.095 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:08.095 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:08.095 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:08.095 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:08.095 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:08.095 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:08.095 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:08.095 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:08.095 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:08.095 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:08.612 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-04-22T07:03:08.615 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-04-22T07:03:08.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-22T07:03:08.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-22T07:03:08.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-22T07:03:08.853 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:08.853 INFO:tasks.ceph:role: osd.1 2024-04-22T07:03:08.853 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-22T07:03:08.853 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-22T07:03:08.924 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:08.924 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:08.925 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:08.925 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:08.925 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:09.378 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-04-22T07:03:09.380 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-04-22T07:03:09.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-22T07:03:09.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-22T07:03:09.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-22T07:03:09.560 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:09.560 INFO:tasks.ceph:role: osd.2 2024-04-22T07:03:09.560 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-22T07:03:09.560 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-22T07:03:09.632 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:09.632 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:09.632 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:09.632 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:09.632 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:09.632 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:09.632 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:09.632 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:09.633 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:09.633 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:10.119 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-04-22T07:03:10.121 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-04-22T07:03:10.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-22T07:03:10.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-22T07:03:10.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-22T07:03:10.342 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:10.342 INFO:tasks.ceph:role: osd.3 2024-04-22T07:03:10.342 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-22T07:03:10.342 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-22T07:03:10.414 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:10.414 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:10.414 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:10.414 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:10.414 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:10.415 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:10.943 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-04-22T07:03:10.945 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-04-22T07:03:10.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-22T07:03:11.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-22T07:03:11.097 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:11.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:11.193+0000 7f1816813640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-22T07:03:11.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:11.193+0000 7f1816813640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-22T07:03:11.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:11.193+0000 7f1816813640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T07:03:11.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:11.193+0000 7f1816813640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T07:03:11.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:11.193+0000 7f1816813640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T07:03:11.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:11.194+0000 7f1816813640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-22T07:03:14.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T07:03:14.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:14.155+0000 7f9b6a8fb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-22T07:03:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:14.155+0000 7f9b6a8fb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-22T07:03:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:14.155+0000 7f9b6a8fb640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T07:03:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:14.155+0000 7f9b6a8fb640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T07:03:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:14.155+0000 7f9b6a8fb640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T07:03:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:14.156+0000 7f9b6a8fb640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-22T07:03:16.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T07:03:17.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:17.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:17.112+0000 7fcdc5db8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-22T07:03:17.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:17.112+0000 7fcdc5db8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-22T07:03:17.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:17.112+0000 7fcdc5db8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T07:03:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:17.112+0000 7fcdc5db8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T07:03:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:17.112+0000 7fcdc5db8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T07:03:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:17.113+0000 7fcdc5db8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-22T07:03:19.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T07:03:19.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:20.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:20.069+0000 7f73132d5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-22T07:03:20.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:20.069+0000 7f73132d5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-22T07:03:20.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:20.069+0000 7f73132d5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T07:03:20.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:20.069+0000 7f73132d5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T07:03:20.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:20.069+0000 7f73132d5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T07:03:20.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:20.070+0000 7f73132d5640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-22T07:03:22.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T07:03:22.986 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-22T07:03:23.014 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:23.015 INFO:tasks.ceph:role: osd.4 2024-04-22T07:03:23.015 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi145.front.sepia.ceph.com 2024-04-22T07:03:23.015 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-22T07:03:23.089 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:23.089 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:23.089 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:23.089 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:23.089 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:23.089 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:23.089 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:23.090 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:23.090 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:23.090 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:23.578 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2024-04-22T07:03:23.580 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2024-04-22T07:03:23.580 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-22T07:03:23.691 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-22T07:03:23.723 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-22T07:03:23.787 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:23.788 INFO:tasks.ceph:role: osd.5 2024-04-22T07:03:23.788 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi145.front.sepia.ceph.com 2024-04-22T07:03:23.788 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-22T07:03:23.872 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:23.872 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:23.872 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:23.872 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:23.872 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:23.872 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:23.872 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:23.872 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:23.872 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:23.873 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:24.330 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2024-04-22T07:03:24.332 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2024-04-22T07:03:24.332 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-22T07:03:24.425 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-22T07:03:24.452 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-22T07:03:24.515 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:24.515 INFO:tasks.ceph:role: osd.6 2024-04-22T07:03:24.515 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi145.front.sepia.ceph.com 2024-04-22T07:03:24.516 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-22T07:03:24.586 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:24.586 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:24.587 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:24.587 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:24.587 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:25.061 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2024-04-22T07:03:25.063 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2024-04-22T07:03:25.063 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-22T07:03:25.190 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-22T07:03:25.222 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-22T07:03:25.286 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:25.287 INFO:tasks.ceph:role: osd.7 2024-04-22T07:03:25.287 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi145.front.sepia.ceph.com 2024-04-22T07:03:25.287 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-22T07:03:25.358 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:25.358 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:25.358 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:25.358 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:25.358 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:25.358 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:25.358 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:25.358 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:25.358 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:25.358 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:25.857 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2024-04-22T07:03:25.859 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2024-04-22T07:03:25.859 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-22T07:03:25.978 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-22T07:03:26.009 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:26.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:26.099+0000 7fb2f060c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-22T07:03:26.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:26.100+0000 7fb2f060c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-22T07:03:26.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:26.100+0000 7fb2f060c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T07:03:26.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:26.100+0000 7fb2f060c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T07:03:26.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:26.100+0000 7fb2f060c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T07:03:26.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:26.100+0000 7fb2f060c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-22T07:03:28.945 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T07:03:28.973 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:29.062 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:29.063+0000 7f1f56015640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-22T07:03:29.062 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:29.063+0000 7f1f56015640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-22T07:03:29.062 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:29.063+0000 7f1f56015640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T07:03:29.062 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:29.063+0000 7f1f56015640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T07:03:29.062 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:29.063+0000 7f1f56015640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T07:03:29.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:29.064+0000 7f1f56015640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-22T07:03:31.908 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T07:03:31.936 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:32.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:32.025+0000 7f38a3c30640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-22T07:03:32.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:32.026+0000 7f38a3c30640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-22T07:03:32.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:32.026+0000 7f38a3c30640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T07:03:32.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:32.026+0000 7f38a3c30640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T07:03:32.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:32.026+0000 7f38a3c30640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T07:03:32.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:32.026+0000 7f38a3c30640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-22T07:03:34.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:03:34.842 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:03:34.876 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-22T07:03:34.979 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:35.065 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:35.067+0000 7f0160ab1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-22T07:03:35.065 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:35.067+0000 7f0160ab1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-22T07:03:35.065 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:35.067+0000 7f0160ab1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T07:03:35.065 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:35.067+0000 7f0160ab1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T07:03:35.065 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:35.067+0000 7f0160ab1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T07:03:35.065 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:35.068+0000 7f0160ab1640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-22T07:03:37.916 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-22T07:03:37.985 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-22T07:03:37.985 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:37.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-22T07:03:38.014 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:38.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-22T07:03:38.081 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:38.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-22T07:03:38.146 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:38.146 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-04-22T07:03:38.211 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:38.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-22T07:03:38.276 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:38.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-22T07:03:38.340 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:38.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-22T07:03:38.404 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:38.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-22T07:03:38.468 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:38.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-22T07:03:38.497 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:38.497 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-22T07:03:38.565 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:38.565 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-22T07:03:38.631 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:38.631 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-04-22T07:03:38.697 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:38.697 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-22T07:03:38.761 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:38.761 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-22T07:03:38.826 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:38.826 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-22T07:03:38.890 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:38.890 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-22T07:03:38.953 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:38.954 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-22T07:03:38.970 INFO:tasks.ceph:Adding keys to all mons... 2024-04-22T07:03:38.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-22T07:03:39.012 DEBUG:teuthology.orchestra.run.smithi145:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-22T07:03:39.033 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2024-04-22T07:03:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCrCyZmBVQUDhAAHls1PS6nxqfEeCHWwzyIyg== 2024-04-22T07:03:39.034 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2024-04-22T07:03:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCrCyZmZ3RfFRAAaBRrV0CMGEzuEYqEtRQiGA== 2024-04-22T07:03:39.034 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2024-04-22T07:03:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCrCyZmu2osHBAAEOP3tW3tfZIH/x1TwziU8w== 2024-04-22T07:03:39.034 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.e] 2024-04-22T07:03:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCrCyZmom37IhAA1oPTZmyEOpLyJRix1f+oog== 2024-04-22T07:03:39.034 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2024-04-22T07:03:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCvCyZmm2qTCxAAKWd788Uz2rX/6VTjeisxHw== 2024-04-22T07:03:39.034 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2024-04-22T07:03:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCyCyZmawljCRAACaWEXmDr6ZGT4KX9T5e7pA== 2024-04-22T07:03:39.034 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQC1CyZmylK/BhAARGdzdcbnDD7bhrRZkMGgFg== 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQC4CyZmqAY0BBAAg8IOSoq1d1DK4JyJnxuY5A== 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCrCyZmWD1IEhAATIjWWfbvj+8UagWhx5aZIw== 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCrCyZmdeJmJxAA5w8c34Lai5qwt+J9ct9wFA== 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCrCyZmGfIjLhAAz+dXi6kO2eimDzfFJvvTWw== 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.f] 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCrCyZmG7ThNBAARrDXfIcAWvaKPMMYpOxK7g== 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQC+CyZmpRQMBhAAc2yN8ageZ/ay3lC8gOiRWg== 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2024-04-22T07:03:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDBCyZmxlzbAxAAIeEI9k/NoJbVHLCi6FDVFw== 2024-04-22T07:03:39.036 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2024-04-22T07:03:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDECyZmJCegARAAEsbqdHX171nMrsW4Ok4SRw== 2024-04-22T07:03:39.036 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2024-04-22T07:03:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDHCyZm4t0JBBAA2BxLqigCNlghECRpgmSkRw== 2024-04-22T07:03:39.036 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2024-04-22T07:03:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCrCyZmIIDKOBAAzPkoESyKmwwYAbNi8hos2g== 2024-04-22T07:03:39.037 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr.y] 2024-04-22T07:03:39.037 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQCrCyZmBVQUDhAAHls1PS6nxqfEeCHWwzyIyg== 2024-04-22T07:03:39.037 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.a] 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQCrCyZmZ3RfFRAAaBRrV0CMGEzuEYqEtRQiGA== 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.c] 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQCrCyZmu2osHBAAEOP3tW3tfZIH/x1TwziU8w== 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.e] 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQCrCyZmom37IhAA1oPTZmyEOpLyJRix1f+oog== 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.0] 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQCvCyZmm2qTCxAAKWd788Uz2rX/6VTjeisxHw== 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.1] 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQCyCyZmawljCRAACaWEXmDr6ZGT4KX9T5e7pA== 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.2] 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQC1CyZmylK/BhAARGdzdcbnDD7bhrRZkMGgFg== 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.3] 2024-04-22T07:03:39.038 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQC4CyZmqAY0BBAAg8IOSoq1d1DK4JyJnxuY5A== 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr.x] 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQCrCyZmWD1IEhAATIjWWfbvj+8UagWhx5aZIw== 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.b] 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQCrCyZmdeJmJxAA5w8c34Lai5qwt+J9ct9wFA== 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.d] 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQCrCyZmGfIjLhAAz+dXi6kO2eimDzfFJvvTWw== 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.f] 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQCrCyZmG7ThNBAARrDXfIcAWvaKPMMYpOxK7g== 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.4] 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQC+CyZmpRQMBhAAc2yN8ageZ/ay3lC8gOiRWg== 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.5] 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDBCyZmxlzbAxAAIeEI9k/NoJbVHLCi6FDVFw== 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.6] 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDECyZmJCegARAAEsbqdHX171nMrsW4Ok4SRw== 2024-04-22T07:03:39.039 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.7] 2024-04-22T07:03:39.040 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDHCyZm4t0JBBAA2BxLqigCNlghECRpgmSkRw== 2024-04-22T07:03:39.040 INFO:teuthology.orchestra.run.smithi145.stdout:[client.0] 2024-04-22T07:03:39.040 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQCrCyZmIIDKOBAAzPkoESyKmwwYAbNi8hos2g== 2024-04-22T07:03:39.040 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T07:03:39.081 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T07:03:39.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:39.133 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:39.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:39.186 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:39.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:39.240 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:39.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.293 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.344 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.394 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.445 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T07:03:39.495 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T07:03:39.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:39.545 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:39.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:39.599 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:39.651 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:39.654 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:39.705 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.707 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.758 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.809 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.860 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:39.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T07:03:39.910 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T07:03:39.961 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-22T07:03:39.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-22T07:03:39.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T07:03:40.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-22T07:03:40.101 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-22T07:03:40.124 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T07:03:40.218 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-22T07:03:40.241 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-22T07:03:40.304 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T07:03:40.398 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-22T07:03:40.421 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:40.427 DEBUG:teuthology.orchestra.run.smithi145:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:40.476 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-22T07:03:40.476 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-22T07:03:40.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-22T07:03:40.479 INFO:tasks.ceph.mon.a:Started 2024-04-22T07:03:40.479 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-22T07:03:40.479 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-22T07:03:40.518 INFO:tasks.ceph.mon.b:Started 2024-04-22T07:03:40.519 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-22T07:03:40.519 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-22T07:03:40.521 INFO:tasks.ceph.mon.c:Started 2024-04-22T07:03:40.521 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-22T07:03:40.521 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-22T07:03:40.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-22T07:03:40.524 INFO:tasks.ceph.mgr.y:Started 2024-04-22T07:03:40.524 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-22T07:03:40.524 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-22T07:03:40.525 INFO:tasks.ceph.mgr.x:Started 2024-04-22T07:03:40.526 INFO:tasks.ceph:Setting crush tunables to default 2024-04-22T07:03:40.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd crush tunables default 2024-04-22T07:03:40.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.626+0000 7ffb76792640 1 Processor -- start 2024-04-22T07:03:40.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.626+0000 7ffb76792640 1 -- start start 2024-04-22T07:03:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb76792640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb76792640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb76792640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb7015ab30 con 0x7ffb700d0f50 2024-04-22T07:03:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb76792640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7ffb7015ba00 con 0x7ffb7015e9b0 2024-04-22T07:03:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb76792640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ffb7015adf0 con 0x7ffb700d0b70 2024-04-22T07:03:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb74d2f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb74d2f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48126/0 (socket says 172.21.15.92:48126) 2024-04-22T07:03:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb74d2f640 1 -- 172.21.15.92:0/3441576153 learned_addr learned my addr 172.21.15.92:0/3441576153 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:03:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb74d2f640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 msgr2=0x7ffb70154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T07:03:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb74d2f640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 msgr2=0x7ffb70154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:03:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 msgr2=0x7ffb70154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T07:03:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 msgr2=0x7ffb70154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.627+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:03:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.828+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.828+0000 7ffb74d2f640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 msgr2=0x7ffb70154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T07:03:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.828+0000 7ffb74d2f640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 msgr2=0x7ffb70154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.828+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.828+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:03:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.828+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.828+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 msgr2=0x7ffb70154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T07:03:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.828+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 msgr2=0x7ffb70154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.828+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:40.828+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:03:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:41.229+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:41.229+0000 7ffb74d2f640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 msgr2=0x7ffb70154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T07:03:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:41.229+0000 7ffb74d2f640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 msgr2=0x7ffb70154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:41.229+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:41.229+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:03:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:41.229+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:41.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:41.229+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 msgr2=0x7ffb70154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T07:03:41.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:41.229+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 msgr2=0x7ffb70154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:41.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:41.229+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:41.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:41.229+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:03:42.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:42.030+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:42.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:42.030+0000 7ffb74d2f640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 msgr2=0x7ffb70154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T07:03:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:42.030+0000 7ffb74d2f640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 msgr2=0x7ffb70154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:42.030+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:42.030+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:03:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:42.030+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:42.030+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 msgr2=0x7ffb70154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T07:03:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:42.030+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 msgr2=0x7ffb70154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:42.030+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:42.030+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:03:43.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6dffb640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 msgr2=0x7ffb70154a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:03:43.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6dffb640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.145:3300/0 conn(0x7ffb700d0b70 0x7ffb70154a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:43.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6dffb640 1 -- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 msgr2=0x7ffb70154f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:03:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6dffb640 1 --2- 172.21.15.92:0/3441576153 >> v2:172.21.15.92:3300/0 conn(0x7ffb7015e9b0 0x7ffb70154f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6dffb640 1 -- 172.21.15.92:0/3441576153 >> v1:172.21.15.145:6789/0 conn(0x7ffb700d0f50 legacy=0x7ffb7015e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6dffb640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6dffb640 1 -- 172.21.15.92:0/3441576153 --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb7015b0d0 con 0x7ffb5c006170 2024-04-22T07:03:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6dffb640 1 -- 172.21.15.92:0/3441576153 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7015ba00 con 0x7ffb5c003690 2024-04-22T07:03:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 msgr2=0x7ffb5c005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:03:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 msgr2=0x7ffb5c005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:43.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:43.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.628+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:03:43.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.829+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:43.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.829+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 msgr2=0x7ffb5c005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:03:43.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.829+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 msgr2=0x7ffb5c005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:43.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.829+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:43.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:43.829+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:03:44.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:44.230+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:44.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:44.230+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 msgr2=0x7ffb5c005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:03:44.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:44.230+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 msgr2=0x7ffb5c005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:44.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:44.230+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:44.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:44.230+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:03:45.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.032+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:45.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.032+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 msgr2=0x7ffb5c005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:03:45.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.032+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 msgr2=0x7ffb5c005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:45.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.032+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:45.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.032+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:03:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2520783231 0 0) 0x7ffb7015b0d0 con 0x7ffb5c006170 2024-04-22T07:03:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb58003520 con 0x7ffb5c006170 2024-04-22T07:03:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1582482346 0 0) 0x7ffb58003520 con 0x7ffb5c006170 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 msgr2=0x7ffb5c005ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb5c003690 0x7ffb5c005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5c00b010 con 0x7ffb5c006170 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7ffb640016f0 con 0x7ffb5c006170 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 >> v1:172.21.15.92:6789/0 conn(0x7ffb5c006170 legacy=0x7ffb5c008610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb74d2f640 1 -- 172.21.15.92:0/3441576153 reap_dead start 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb580039b0 0x7ffb58003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb58004800 0x7ffb58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb58005290 0x7ffb5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7015ba00 con 0x7ffb580039b0 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5800c1e0 con 0x7ffb58005290 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5800c3e0 con 0x7ffb58004800 2024-04-22T07:03:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb580039b0 0x7ffb58003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.798+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb58005290 0x7ffb5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.799+0000 7ffb6f7fe640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb58004800 0x7ffb58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.799+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb58004800 msgr2=0x7ffb58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.799+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb58004800 0x7ffb58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.799+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb58005290 msgr2=0x7ffb5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.799+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb58005290 0x7ffb5800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.799+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3441576153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb58003520 con 0x7ffb580039b0 2024-04-22T07:03:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.800+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb580039b0 0x7ffb58003db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb60019cc0 tx=0x7ffb58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0394f0f841228c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.800+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60019940 con 0x7ffb580039b0 2024-04-22T07:03:45.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.800+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb6002f500 con 0x7ffb580039b0 2024-04-22T07:03:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.800+0000 7ffb6effd640 1 -- 172.21.15.92:0/3441576153 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6002f910 con 0x7ffb580039b0 2024-04-22T07:03:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.801+0000 7ffb76792640 1 -- 172.21.15.92:0/3441576153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb580039b0 msgr2=0x7ffb58003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.801+0000 7ffb76792640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb580039b0 0x7ffb58003db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb60019cc0 tx=0x7ffb58005830 comp rx=0 tx=0).stop 2024-04-22T07:03:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.801+0000 7ffb76792640 1 -- 172.21.15.92:0/3441576153 shutdown_connections 2024-04-22T07:03:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.801+0000 7ffb76792640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb58005290 0x7ffb5800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.801+0000 7ffb76792640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb58004800 0x7ffb58004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.801+0000 7ffb76792640 1 --2- 172.21.15.92:0/3441576153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb580039b0 0x7ffb58003db0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.801+0000 7ffb76792640 1 -- 172.21.15.92:0/3441576153 >> 172.21.15.92:0/3441576153 conn(0x7ffb700736b0 msgr2=0x7ffb70073ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.801+0000 7ffb76792640 1 -- 172.21.15.92:0/3441576153 shutdown_connections 2024-04-22T07:03:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.801+0000 7ffb76792640 1 -- 172.21.15.92:0/3441576153 wait complete. 2024-04-22T07:03:45.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.803+0000 7ffb76792640 1 Processor -- start 2024-04-22T07:03:45.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.803+0000 7ffb76792640 1 -- start start 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.804+0000 7ffb76792640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb580039b0 0x7ffb70157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.804+0000 7ffb76792640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb58004800 0x7ffb700b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.804+0000 7ffb76792640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb58005290 0x7ffb700b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.804+0000 7ffb76792640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffb700cae00 con 0x7ffb58004800 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.804+0000 7ffb76792640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7015b690 con 0x7ffb58005290 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.804+0000 7ffb76792640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffb700caf80 con 0x7ffb580039b0 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.804+0000 7ffb6ffff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb580039b0 0x7ffb70157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.804+0000 7ffb6f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb58004800 0x7ffb700b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.804+0000 7ffb6ffff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb580039b0 0x7ffb70157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:59316/0 (socket says 172.21.15.92:59316) 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.804+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3606681623 learned_addr learned my addr 172.21.15.92:0/3606681623 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.804+0000 7ffb6f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb58004800 0x7ffb700b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48196/0 (socket says 172.21.15.92:48196) 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.804+0000 7ffb74d2f640 1 --2- 172.21.15.92:0/3606681623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb58005290 0x7ffb700b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.805+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3606681623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb58005290 msgr2=0x7ffb700b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:45.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.805+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3606681623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb58005290 0x7ffb700b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:45.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.805+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3606681623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb58004800 msgr2=0x7ffb700b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:45.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.805+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3606681623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb58004800 0x7ffb700b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:45.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.805+0000 7ffb6ffff640 1 -- 172.21.15.92:0/3606681623 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb701584d0 con 0x7ffb580039b0 2024-04-22T07:03:45.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.805+0000 7ffb6f7fe640 1 --2- 172.21.15.92:0/3606681623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb58004800 0x7ffb700b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:45.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.805+0000 7ffb6ffff640 1 --2- 172.21.15.92:0/3606681623 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb580039b0 0x7ffb70157f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb60019db0 tx=0x7ffb600046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:45.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.805+0000 7ffb6cff9640 1 -- 172.21.15.92:0/3606681623 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb600044a0 con 0x7ffb580039b0 2024-04-22T07:03:45.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.805+0000 7ffb6cff9640 1 -- 172.21.15.92:0/3606681623 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb60047070 con 0x7ffb580039b0 2024-04-22T07:03:45.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.805+0000 7ffb76792640 1 -- 172.21.15.92:0/3606681623 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb701f4640 con 0x7ffb580039b0 2024-04-22T07:03:45.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.805+0000 7ffb6cff9640 1 -- 172.21.15.92:0/3606681623 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6002f8c0 con 0x7ffb580039b0 2024-04-22T07:03:45.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.806+0000 7ffb76792640 1 -- 172.21.15.92:0/3606681623 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb701f0940 con 0x7ffb580039b0 2024-04-22T07:03:45.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.806+0000 7ffb6cff9640 1 -- 172.21.15.92:0/3606681623 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffb6002fa60 con 0x7ffb580039b0 2024-04-22T07:03:45.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.806+0000 7ffb6cff9640 1 -- 172.21.15.92:0/3606681623 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb6004c460 con 0x7ffb580039b0 2024-04-22T07:03:45.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.807+0000 7ffb76792640 1 -- 172.21.15.92:0/3606681623 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb70069930 con 0x7ffb580039b0 2024-04-22T07:03:45.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.810+0000 7ffb6cff9640 1 -- 172.21.15.92:0/3606681623 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ffb60003610 con 0x7ffb580039b0 2024-04-22T07:03:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.879+0000 7ffb76792640 1 -- 172.21.15.92:0/3606681623 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ffb70154f20 con 0x7ffb580039b0 2024-04-22T07:03:45.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.883+0000 7ffb6cff9640 1 -- 172.21.15.92:0/3606681623 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7ffb600037f0 con 0x7ffb580039b0 2024-04-22T07:03:45.881 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2024-04-22T07:03:45.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.884+0000 7ffb76792640 1 -- 172.21.15.92:0/3606681623 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb580039b0 msgr2=0x7ffb70157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:45.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.884+0000 7ffb76792640 1 --2- 172.21.15.92:0/3606681623 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb580039b0 0x7ffb70157f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb60019db0 tx=0x7ffb600046c0 comp rx=0 tx=0).stop 2024-04-22T07:03:45.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.884+0000 7ffb76792640 1 -- 172.21.15.92:0/3606681623 shutdown_connections 2024-04-22T07:03:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.884+0000 7ffb76792640 1 --2- 172.21.15.92:0/3606681623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb58005290 0x7ffb700b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.884+0000 7ffb76792640 1 --2- 172.21.15.92:0/3606681623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb58004800 0x7ffb700b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.884+0000 7ffb76792640 1 --2- 172.21.15.92:0/3606681623 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb580039b0 0x7ffb70157f90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.884+0000 7ffb76792640 1 -- 172.21.15.92:0/3606681623 >> 172.21.15.92:0/3606681623 conn(0x7ffb700736b0 msgr2=0x7ffb70073a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.884+0000 7ffb76792640 1 -- 172.21.15.92:0/3606681623 shutdown_connections 2024-04-22T07:03:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:45.884+0000 7ffb76792640 1 -- 172.21.15.92:0/3606681623 wait complete. 2024-04-22T07:03:45.896 INFO:tasks.ceph:check_enable_crimson: False 2024-04-22T07:03:45.896 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-22T07:03:45.897 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:45.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-22T07:03:45.910 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:45.913+0000 7efe53c41040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T07:03:45.924 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:45.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-22T07:03:45.931 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:45.934+0000 7fdde9638040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T07:03:45.956 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:45.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-22T07:03:46.021 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:03:46.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-22T07:03:46.064 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:46.067+0000 7efe53c41040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:03:46.085 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:46.085 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-22T07:03:46.094 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:46.096+0000 7fdde9638040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:03:46.108 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:46.108 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-22T07:03:46.119 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:46.122+0000 7efe53c41040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T07:03:46.141 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:46.142 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-22T07:03:46.148 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:46.150+0000 7fdde9638040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T07:03:46.205 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-22T07:03:46.205 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-22T07:03:46.269 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new d554be68-0854-4df8-b5a3-2d07f668274d 0 2024-04-22T07:03:46.385 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:46.388+0000 7efe53c41040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T07:03:46.402 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:46.404+0000 7fdde9638040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T07:03:46.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.410+0000 7effc8cf4640 1 Processor -- start 2024-04-22T07:03:46.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.410+0000 7effc8cf4640 1 -- start start 2024-04-22T07:03:46.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc8cf4640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7effc40c7fc0 0x7effc415b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc8cf4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc415e5b0 0x7effc415ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc8cf4640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effc415ef60 con 0x7effc4155030 2024-04-22T07:03:46.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc8cf4640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7effc415bcd0 con 0x7effc40c7fc0 2024-04-22T07:03:46.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc8cf4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7effc40c8b00 con 0x7effc415e5b0 2024-04-22T07:03:46.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc415e5b0 0x7effc415ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc415e5b0 0x7effc415ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:35206/0 (socket says 172.21.15.145:35206) 2024-04-22T07:03:46.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc2d9d640 1 -- 172.21.15.145:0/272283438 learned_addr learned my addr 172.21.15.145:0/272283438 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:46.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc1d9b640 1 --2- 172.21.15.145:0/272283438 >> v2:172.21.15.92:3300/0 conn(0x7effc40c7fc0 0x7effc415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc2d9d640 1 -- 172.21.15.145:0/272283438 >> v2:172.21.15.92:3300/0 conn(0x7effc40c7fc0 msgr2=0x7effc415b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:46.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc2d9d640 1 --2- 172.21.15.145:0/272283438 >> v2:172.21.15.92:3300/0 conn(0x7effc40c7fc0 0x7effc415b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc2d9d640 1 -- 172.21.15.145:0/272283438 >> v1:172.21.15.92:6789/0 conn(0x7effc4155030 legacy=0x7effc40c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.411+0000 7effc2d9d640 1 -- 172.21.15.145:0/272283438 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc40c8880 con 0x7effc415e5b0 2024-04-22T07:03:46.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.412+0000 7effc2d9d640 1 --2- 172.21.15.145:0/272283438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc415e5b0 0x7effc415ea20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7effbc009e50 tx=0x7effbc02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6abc11ca977979c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:46.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.412+0000 7effc159a640 1 -- 172.21.15.145:0/272283438 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc02cb00 con 0x7effc415e5b0 2024-04-22T07:03:46.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.412+0000 7effc159a640 1 -- 172.21.15.145:0/272283438 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effbc02cca0 con 0x7effc415e5b0 2024-04-22T07:03:46.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.412+0000 7effc159a640 1 -- 172.21.15.145:0/272283438 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc034610 con 0x7effc415e5b0 2024-04-22T07:03:46.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.413+0000 7effc8cf4640 1 -- 172.21.15.145:0/272283438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc415e5b0 msgr2=0x7effc415ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.413+0000 7effc8cf4640 1 --2- 172.21.15.145:0/272283438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc415e5b0 0x7effc415ea20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7effbc009e50 tx=0x7effbc02bf60 comp rx=0 tx=0).stop 2024-04-22T07:03:46.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.413+0000 7effc8cf4640 1 -- 172.21.15.145:0/272283438 shutdown_connections 2024-04-22T07:03:46.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.413+0000 7effc8cf4640 1 --2- 172.21.15.145:0/272283438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc415e5b0 0x7effc415ea20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.413+0000 7effc8cf4640 1 --2- 172.21.15.145:0/272283438 >> v2:172.21.15.92:3300/0 conn(0x7effc40c7fc0 0x7effc415b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.413+0000 7effc8cf4640 1 -- 172.21.15.145:0/272283438 >> 172.21.15.145:0/272283438 conn(0x7effc406b970 msgr2=0x7effc406bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:46.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.413+0000 7effc8cf4640 1 -- 172.21.15.145:0/272283438 shutdown_connections 2024-04-22T07:03:46.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.413+0000 7effc8cf4640 1 -- 172.21.15.145:0/272283438 wait complete. 2024-04-22T07:03:46.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.414+0000 7effc8cf4640 1 Processor -- start 2024-04-22T07:03:46.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.414+0000 7effc8cf4640 1 -- start start 2024-04-22T07:03:46.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc8cf4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc40c7fc0 0x7effc414f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc8cf4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effc4155030 0x7effc41f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc8cf4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc415e5b0 0x7effc41f1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc8cf4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7effc40c75f0 con 0x7effc415e5b0 2024-04-22T07:03:46.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc8cf4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7effc40c9030 con 0x7effc4155030 2024-04-22T07:03:46.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc8cf4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7effc40c78d0 con 0x7effc40c7fc0 2024-04-22T07:03:46.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc40c7fc0 0x7effc414f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc1d9b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effc4155030 0x7effc41f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc40c7fc0 0x7effc414f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:35218/0 (socket says 172.21.15.145:35218) 2024-04-22T07:03:46.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc259c640 1 -- 172.21.15.145:0/258416905 learned_addr learned my addr 172.21.15.145:0/258416905 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc2d9d640 1 --2- 172.21.15.145:0/258416905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc415e5b0 0x7effc41f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc259c640 1 -- 172.21.15.145:0/258416905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effc4155030 msgr2=0x7effc41f0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc259c640 1 --2- 172.21.15.145:0/258416905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effc4155030 0x7effc41f0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc259c640 1 -- 172.21.15.145:0/258416905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc415e5b0 msgr2=0x7effc41f1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc259c640 1 --2- 172.21.15.145:0/258416905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc415e5b0 0x7effc41f1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.415+0000 7effc259c640 1 -- 172.21.15.145:0/258416905 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc40cc150 con 0x7effc40c7fc0 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.416+0000 7effc259c640 1 --2- 172.21.15.145:0/258416905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc40c7fc0 0x7effc414f540 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7effb800c400 tx=0x7effb800c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.416+0000 7effb2ffd640 1 -- 172.21.15.145:0/258416905 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb80052d0 con 0x7effc40c7fc0 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.416+0000 7effb2ffd640 1 -- 172.21.15.145:0/258416905 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb8005490 con 0x7effc40c7fc0 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.416+0000 7effc8cf4640 1 -- 172.21.15.145:0/258416905 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc41533b0 con 0x7effc40c7fc0 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.416+0000 7effb2ffd640 1 -- 172.21.15.145:0/258416905 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8007260 con 0x7effc40c7fc0 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.416+0000 7effc8cf4640 1 -- 172.21.15.145:0/258416905 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7effc40cc3f0 con 0x7effc40c7fc0 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.416+0000 7effb2ffd640 1 -- 172.21.15.145:0/258416905 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7effb8007400 con 0x7effc40c7fc0 2024-04-22T07:03:46.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.416+0000 7effb2ffd640 1 -- 172.21.15.145:0/258416905 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7effb8003b20 con 0x7effc40c7fc0 2024-04-22T07:03:46.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.417+0000 7effc8cf4640 1 -- 172.21.15.145:0/258416905 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc4068b00 con 0x7effc40c7fc0 2024-04-22T07:03:46.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.419+0000 7effb2ffd640 1 -- 172.21.15.145:0/258416905 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7effb80035f0 con 0x7effc40c7fc0 2024-04-22T07:03:46.445 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:46.448+0000 7efe53c41040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T07:03:46.449 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:46.451+0000 7fdde9638040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T07:03:46.552 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.555+0000 7effc8cf4640 1 -- 172.21.15.145:0/258416905 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d554be68-0854-4df8-b5a3-2d07f668274d", "id": 0} v 0) -- 0x7effc40c3970 con 0x7effc40c7fc0 2024-04-22T07:03:46.555 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.558+0000 7effb2ffd640 1 -- 172.21.15.145:0/258416905 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d554be68-0854-4df8-b5a3-2d07f668274d", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7effb8003d20 con 0x7effc40c7fc0 2024-04-22T07:03:46.555 INFO:teuthology.orchestra.run.smithi145.stdout:0 2024-04-22T07:03:46.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.559+0000 7effc8cf4640 1 -- 172.21.15.145:0/258416905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc40c7fc0 msgr2=0x7effc414f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.559+0000 7effc8cf4640 1 --2- 172.21.15.145:0/258416905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc40c7fc0 0x7effc414f540 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7effb800c400 tx=0x7effb800c8c0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.557 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.559+0000 7effc8cf4640 1 -- 172.21.15.145:0/258416905 shutdown_connections 2024-04-22T07:03:46.557 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.559+0000 7effc8cf4640 1 --2- 172.21.15.145:0/258416905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc415e5b0 0x7effc41f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.557 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.559+0000 7effc8cf4640 1 --2- 172.21.15.145:0/258416905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effc4155030 0x7effc41f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.557 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.559+0000 7effc8cf4640 1 --2- 172.21.15.145:0/258416905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7effc40c7fc0 0x7effc414f540 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.557 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.559+0000 7effc8cf4640 1 -- 172.21.15.145:0/258416905 >> 172.21.15.145:0/258416905 conn(0x7effc406b970 msgr2=0x7effc40731f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:46.557 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.559+0000 7effc8cf4640 1 -- 172.21.15.145:0/258416905 shutdown_connections 2024-04-22T07:03:46.557 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.559+0000 7effc8cf4640 1 -- 172.21.15.145:0/258416905 wait complete. 2024-04-22T07:03:46.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 963d3334-a43c-4aaf-b6b3-841b520d6c03 1 2024-04-22T07:03:46.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.666+0000 7f32948eb640 1 Processor -- start 2024-04-22T07:03:46.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.666+0000 7f32948eb640 1 -- start start 2024-04-22T07:03:46.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f32948eb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f329014f290 0x7f3290159e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f32948eb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f32900707e0 0x7f329015a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f32948eb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3290072590 con 0x7f3290072090 2024-04-22T07:03:46.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f32948eb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f3290072850 con 0x7f32900707e0 2024-04-22T07:03:46.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f32948eb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3290158ba0 con 0x7f329014f290 2024-04-22T07:03:46.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f328e59c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f329014f290 0x7f3290159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f328e59c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f329014f290 0x7f3290159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:34224/0 (socket says 172.21.15.145:34224) 2024-04-22T07:03:46.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f328ed9d640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f3290072090 0x7f32900c60f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:57014/0 (socket says 172.21.15.145:57014) 2024-04-22T07:03:46.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f328e59c640 1 -- 172.21.15.145:0/195049804 learned_addr learned my addr 172.21.15.145:0/195049804 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:46.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f328dd9b640 1 --2- 172.21.15.145:0/195049804 >> v2:172.21.15.92:3300/0 conn(0x7f32900707e0 0x7f329015a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f328e59c640 1 -- 172.21.15.145:0/195049804 >> v2:172.21.15.92:3300/0 conn(0x7f32900707e0 msgr2=0x7f329015a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:46.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f328e59c640 1 --2- 172.21.15.145:0/195049804 >> v2:172.21.15.92:3300/0 conn(0x7f32900707e0 0x7f329015a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f328e59c640 1 -- 172.21.15.145:0/195049804 >> v1:172.21.15.145:6789/0 conn(0x7f3290072090 legacy=0x7f32900c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.668+0000 7f328e59c640 1 -- 172.21.15.145:0/195049804 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3290158e80 con 0x7f329014f290 2024-04-22T07:03:46.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.667+0000 7f328d59a640 1 -- 172.21.15.145:0/195049804 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3409218345 0 0) 0x7f3290072590 con 0x7f3290072090 2024-04-22T07:03:46.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.668+0000 7f328e59c640 1 --2- 172.21.15.145:0/195049804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f329014f290 0x7f3290159e80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f32900c7010 tx=0x7f327802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52a63c20f4d39fea server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:46.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.668+0000 7f328d59a640 1 -- 172.21.15.145:0/195049804 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278034070 con 0x7f329014f290 2024-04-22T07:03:46.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.668+0000 7f328d59a640 1 -- 172.21.15.145:0/195049804 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3278030860 con 0x7f329014f290 2024-04-22T07:03:46.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.668+0000 7f328d59a640 1 -- 172.21.15.145:0/195049804 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278030c30 con 0x7f329014f290 2024-04-22T07:03:46.670 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.669+0000 7f32948eb640 1 -- 172.21.15.145:0/195049804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f329014f290 msgr2=0x7f3290159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.670 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.669+0000 7f32948eb640 1 --2- 172.21.15.145:0/195049804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f329014f290 0x7f3290159e80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f32900c7010 tx=0x7f327802b9a0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.670 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.669+0000 7f32948eb640 1 -- 172.21.15.145:0/195049804 shutdown_connections 2024-04-22T07:03:46.670 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.669+0000 7f32948eb640 1 --2- 172.21.15.145:0/195049804 >> v2:172.21.15.92:3300/0 conn(0x7f32900707e0 0x7f329015a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.670 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.669+0000 7f32948eb640 1 --2- 172.21.15.145:0/195049804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f329014f290 0x7f3290159e80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.670 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.669+0000 7f32948eb640 1 -- 172.21.15.145:0/195049804 >> 172.21.15.145:0/195049804 conn(0x7f329006b970 msgr2=0x7f329006bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:46.670 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.669+0000 7f32948eb640 1 -- 172.21.15.145:0/195049804 shutdown_connections 2024-04-22T07:03:46.670 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.669+0000 7f32948eb640 1 -- 172.21.15.145:0/195049804 wait complete. 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.670+0000 7f32948eb640 1 Processor -- start 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.670+0000 7f32948eb640 1 -- start start 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.671+0000 7f32948eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32900707e0 0x7f3290147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.671+0000 7f32948eb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3290072090 0x7f329015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.671+0000 7f32948eb640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f329014f290 0x7f329015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.671+0000 7f32948eb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3290153030 con 0x7f32900707e0 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.671+0000 7f32948eb640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3290159440 con 0x7f329014f290 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.671+0000 7f32948eb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f32900c6800 con 0x7f3290072090 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.671+0000 7f328ed9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f329014f290 0x7f329015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.671+0000 7f328dd9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3290072090 0x7f329015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.671+0000 7f328ed9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f329014f290 0x7f329015b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:34230/0 (socket says 172.21.15.145:34230) 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.671+0000 7f328ed9d640 1 -- 172.21.15.145:0/3139333653 learned_addr learned my addr 172.21.15.145:0/3139333653 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.671+0000 7f328e59c640 1 --2- 172.21.15.145:0/3139333653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32900707e0 0x7f3290147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f328ed9d640 1 -- 172.21.15.145:0/3139333653 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3290072090 msgr2=0x7f329015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f328ed9d640 1 --2- 172.21.15.145:0/3139333653 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3290072090 0x7f329015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f328ed9d640 1 -- 172.21.15.145:0/3139333653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32900707e0 msgr2=0x7f3290147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f328ed9d640 1 --2- 172.21.15.145:0/3139333653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32900707e0 0x7f3290147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f328ed9d640 1 -- 172.21.15.145:0/3139333653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329014d640 con 0x7f329014f290 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f328dd9b640 1 --2- 172.21.15.145:0/3139333653 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3290072090 0x7f329015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f328e59c640 1 --2- 172.21.15.145:0/3139333653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32900707e0 0x7f3290147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f328ed9d640 1 --2- 172.21.15.145:0/3139333653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f329014f290 0x7f329015b2f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f328800c7a0 tx=0x7f328800cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f3286ffd640 1 -- 172.21.15.145:0/3139333653 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288018070 con 0x7f329014f290 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f3286ffd640 1 -- 172.21.15.145:0/3139333653 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328800a640 con 0x7f329014f290 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f32948eb640 1 -- 172.21.15.145:0/3139333653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329014e0f0 con 0x7f329014f290 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f3286ffd640 1 -- 172.21.15.145:0/3139333653 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328800aa70 con 0x7f329014f290 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f32948eb640 1 -- 172.21.15.145:0/3139333653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329014db90 con 0x7f329014f290 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f3286ffd640 1 -- 172.21.15.145:0/3139333653 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f328801b070 con 0x7f329014f290 2024-04-22T07:03:46.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.672+0000 7f3286ffd640 1 -- 172.21.15.145:0/3139333653 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f328801d7f0 con 0x7f329014f290 2024-04-22T07:03:46.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.673+0000 7f32948eb640 1 -- 172.21.15.145:0/3139333653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3290068b00 con 0x7f329014f290 2024-04-22T07:03:46.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.675+0000 7f3286ffd640 1 -- 172.21.15.145:0/3139333653 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f32880078a0 con 0x7f329014f290 2024-04-22T07:03:46.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.747+0000 7f32948eb640 1 -- 172.21.15.145:0/3139333653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd new", "uuid": "963d3334-a43c-4aaf-b6b3-841b520d6c03", "id": 1} v 0) -- 0x7f329015b9a0 con 0x7f329014f290 2024-04-22T07:03:46.750 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.753+0000 7f3286ffd640 1 -- 172.21.15.145:0/3139333653 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "963d3334-a43c-4aaf-b6b3-841b520d6c03", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f328800ac10 con 0x7f329014f290 2024-04-22T07:03:46.750 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-04-22T07:03:46.751 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.753+0000 7f32948eb640 1 -- 172.21.15.145:0/3139333653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f329014f290 msgr2=0x7f329015b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.751 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.753+0000 7f32948eb640 1 --2- 172.21.15.145:0/3139333653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f329014f290 0x7f329015b2f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f328800c7a0 tx=0x7f328800cc60 comp rx=0 tx=0).stop 2024-04-22T07:03:46.751 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.753+0000 7f32948eb640 1 -- 172.21.15.145:0/3139333653 shutdown_connections 2024-04-22T07:03:46.751 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.753+0000 7f32948eb640 1 --2- 172.21.15.145:0/3139333653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f329014f290 0x7f329015b2f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.753+0000 7f32948eb640 1 --2- 172.21.15.145:0/3139333653 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3290072090 0x7f329015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.753+0000 7f32948eb640 1 --2- 172.21.15.145:0/3139333653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32900707e0 0x7f3290147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.753+0000 7f32948eb640 1 -- 172.21.15.145:0/3139333653 >> 172.21.15.145:0/3139333653 conn(0x7f329006b970 msgr2=0x7f32900c7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:46.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.753+0000 7f32948eb640 1 -- 172.21.15.145:0/3139333653 shutdown_connections 2024-04-22T07:03:46.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.754+0000 7f32948eb640 1 -- 172.21.15.145:0/3139333653 wait complete. 2024-04-22T07:03:46.761 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 480ebbfe-df23-4a47-98b4-f46d6b4f4bb2 2 2024-04-22T07:03:46.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.861+0000 7f463c19f640 1 Processor -- start 2024-04-22T07:03:46.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.861+0000 7f463c19f640 1 -- start start 2024-04-22T07:03:46.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.862+0000 7f463c19f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 0x7f4634070060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.862+0000 7f463c19f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f46340720c0 0x7f463406c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.862+0000 7f463c19f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46340c9dc0 con 0x7f46340705a0 2024-04-22T07:03:46.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.862+0000 7f463c19f640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f46340ca700 con 0x7f46340720c0 2024-04-22T07:03:46.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.862+0000 7f463c19f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f463406f8f0 con 0x7f4634154e90 2024-04-22T07:03:46.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.862+0000 7f4639f3b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 0x7f4634070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.862+0000 7f4639f3b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 0x7f4634070060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:35228/0 (socket says 172.21.15.145:35228) 2024-04-22T07:03:46.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.862+0000 7f4639f3b640 1 -- 172.21.15.145:0/840026317 learned_addr learned my addr 172.21.15.145:0/840026317 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:46.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.863+0000 7f463973a640 1 --2- 172.21.15.145:0/840026317 >> v2:172.21.15.92:3300/0 conn(0x7f46340720c0 0x7f463406c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.863+0000 7f4639f3b640 1 -- 172.21.15.145:0/840026317 >> v2:172.21.15.92:3300/0 conn(0x7f46340720c0 msgr2=0x7f463406c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:46.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.863+0000 7f4639f3b640 1 --2- 172.21.15.145:0/840026317 >> v2:172.21.15.92:3300/0 conn(0x7f46340720c0 0x7f463406c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.863+0000 7f4639f3b640 1 -- 172.21.15.145:0/840026317 >> v1:172.21.15.92:6789/0 conn(0x7f46340705a0 legacy=0x7f463415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.863+0000 7f4639f3b640 1 -- 172.21.15.145:0/840026317 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463415ef90 con 0x7f4634154e90 2024-04-22T07:03:46.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.863+0000 7f4639f3b640 1 --2- 172.21.15.145:0/840026317 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 0x7f4634070060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4630009870 tx=0x7f463002ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e10773c05c666e75 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:46.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.864+0000 7f4638f39640 1 -- 172.21.15.145:0/840026317 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463002f840 con 0x7f4634154e90 2024-04-22T07:03:46.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.864+0000 7f4638f39640 1 -- 172.21.15.145:0/840026317 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f463002f9e0 con 0x7f4634154e90 2024-04-22T07:03:46.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.864+0000 7f4638f39640 1 -- 172.21.15.145:0/840026317 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463002fdb0 con 0x7f4634154e90 2024-04-22T07:03:46.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.864+0000 7f463c19f640 1 -- 172.21.15.145:0/840026317 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 msgr2=0x7f4634070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.864+0000 7f463c19f640 1 --2- 172.21.15.145:0/840026317 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 0x7f4634070060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4630009870 tx=0x7f463002ba20 comp rx=0 tx=0).stop 2024-04-22T07:03:46.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.864+0000 7f463c19f640 1 -- 172.21.15.145:0/840026317 shutdown_connections 2024-04-22T07:03:46.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.864+0000 7f463c19f640 1 --2- 172.21.15.145:0/840026317 >> v2:172.21.15.92:3300/0 conn(0x7f46340720c0 0x7f463406c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.864+0000 7f463c19f640 1 --2- 172.21.15.145:0/840026317 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 0x7f4634070060 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.864+0000 7f463c19f640 1 -- 172.21.15.145:0/840026317 >> 172.21.15.145:0/840026317 conn(0x7f463406e210 msgr2=0x7f4634071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:46.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.864+0000 7f463c19f640 1 -- 172.21.15.145:0/840026317 shutdown_connections 2024-04-22T07:03:46.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.864+0000 7f463c19f640 1 -- 172.21.15.145:0/840026317 wait complete. 2024-04-22T07:03:46.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.865+0000 7f463c19f640 1 Processor -- start 2024-04-22T07:03:46.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.865+0000 7f463c19f640 1 -- start start 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.866+0000 7f463c19f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46340705a0 0x7f46341f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.866+0000 7f463c19f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46340720c0 0x7f4634157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.866+0000 7f463c19f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 0x7f46340c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.866+0000 7f463c19f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f46340ca360 con 0x7f46340720c0 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.866+0000 7f463c19f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f463406fbd0 con 0x7f46340705a0 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.866+0000 7f463c19f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4634155270 con 0x7f4634154e90 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.866+0000 7f4639f3b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46340705a0 0x7f46341f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.866+0000 7f463a73c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 0x7f46340c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.866+0000 7f4639f3b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46340705a0 0x7f46341f3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:34236/0 (socket says 172.21.15.145:34236) 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.866+0000 7f4639f3b640 1 -- 172.21.15.145:0/2428944689 learned_addr learned my addr 172.21.15.145:0/2428944689 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.866+0000 7f463973a640 1 --2- 172.21.15.145:0/2428944689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46340720c0 0x7f4634157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f463a73c640 1 -- 172.21.15.145:0/2428944689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46340705a0 msgr2=0x7f46341f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f463a73c640 1 --2- 172.21.15.145:0/2428944689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46340705a0 0x7f46341f3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f463a73c640 1 -- 172.21.15.145:0/2428944689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46340720c0 msgr2=0x7f4634157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f463a73c640 1 --2- 172.21.15.145:0/2428944689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46340720c0 0x7f4634157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f463a73c640 1 -- 172.21.15.145:0/2428944689 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46341556c0 con 0x7f4634154e90 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f4639f3b640 1 --2- 172.21.15.145:0/2428944689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46340705a0 0x7f46341f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f463973a640 1 --2- 172.21.15.145:0/2428944689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46340720c0 0x7f4634157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f463a73c640 1 --2- 172.21.15.145:0/2428944689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 0x7f46340c47e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f462c00c6c0 tx=0x7f462c00cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f46267fc640 1 -- 172.21.15.145:0/2428944689 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c003b70 con 0x7f4634154e90 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f46267fc640 1 -- 172.21.15.145:0/2428944689 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f462c003d10 con 0x7f4634154e90 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f463c19f640 1 -- 172.21.15.145:0/2428944689 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46341559a0 con 0x7f4634154e90 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f46267fc640 1 -- 172.21.15.145:0/2428944689 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c0052f0 con 0x7f4634154e90 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f463c19f640 1 -- 172.21.15.145:0/2428944689 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4634156850 con 0x7f4634154e90 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f46267fc640 1 -- 172.21.15.145:0/2428944689 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f462c005490 con 0x7f4634154e90 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.867+0000 7f46267fc640 1 -- 172.21.15.145:0/2428944689 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f462c012920 con 0x7f4634154e90 2024-04-22T07:03:46.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.868+0000 7f463c19f640 1 -- 172.21.15.145:0/2428944689 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463406cc10 con 0x7f4634154e90 2024-04-22T07:03:46.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.870+0000 7f46267fc640 1 -- 172.21.15.145:0/2428944689 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f463406cc10 con 0x7f4634154e90 2024-04-22T07:03:46.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.943+0000 7f463c19f640 1 -- 172.21.15.145:0/2428944689 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd new", "uuid": "480ebbfe-df23-4a47-98b4-f46d6b4f4bb2", "id": 2} v 0) -- 0x7f4634068b00 con 0x7f4634154e90 2024-04-22T07:03:46.946 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.949+0000 7f46267fc640 1 -- 172.21.15.145:0/2428944689 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "480ebbfe-df23-4a47-98b4-f46d6b4f4bb2", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f462c012500 con 0x7f4634154e90 2024-04-22T07:03:46.946 INFO:teuthology.orchestra.run.smithi145.stdout:2 2024-04-22T07:03:46.947 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.950+0000 7f463c19f640 1 -- 172.21.15.145:0/2428944689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 msgr2=0x7f46340c47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:46.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.950+0000 7f463c19f640 1 --2- 172.21.15.145:0/2428944689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 0x7f46340c47e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f462c00c6c0 tx=0x7f462c00cb80 comp rx=0 tx=0).stop 2024-04-22T07:03:46.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.950+0000 7f463c19f640 1 -- 172.21.15.145:0/2428944689 shutdown_connections 2024-04-22T07:03:46.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.950+0000 7f463c19f640 1 --2- 172.21.15.145:0/2428944689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634154e90 0x7f46340c47e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.950+0000 7f463c19f640 1 --2- 172.21.15.145:0/2428944689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46340720c0 0x7f4634157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.950+0000 7f463c19f640 1 --2- 172.21.15.145:0/2428944689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46340705a0 0x7f46341f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:46.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.950+0000 7f463c19f640 1 -- 172.21.15.145:0/2428944689 >> 172.21.15.145:0/2428944689 conn(0x7f463406e210 msgr2=0x7f463406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:46.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.950+0000 7f463c19f640 1 -- 172.21.15.145:0/2428944689 shutdown_connections 2024-04-22T07:03:46.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:46.950+0000 7f463c19f640 1 -- 172.21.15.145:0/2428944689 wait complete. 2024-04-22T07:03:46.957 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 3edb4281-5113-4f74-930f-bf39cfae1d14 3 2024-04-22T07:03:47.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.066+0000 7fd7439f4640 1 Processor -- start 2024-04-22T07:03:47.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.066+0000 7fd7439f4640 1 -- start start 2024-04-22T07:03:47.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.067+0000 7fd7439f4640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd73c156220 0x7fd73c070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.067+0000 7fd7439f4640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd73c0c8df0 0x7fd73c159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.067+0000 7fd7439f4640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73c0c3410 con 0x7fd73c0c9e50 2024-04-22T07:03:47.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.067+0000 7fd7439f4640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fd73c06ffe0 con 0x7fd73c156220 2024-04-22T07:03:47.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.067+0000 7fd7439f4640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd73c0c3990 con 0x7fd73c0c8df0 2024-04-22T07:03:47.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.067+0000 7fd740f8f640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd73c0c8df0 0x7fd73c159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.067+0000 7fd740f8f640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd73c0c8df0 0x7fd73c159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:34252/0 (socket says 172.21.15.145:34252) 2024-04-22T07:03:47.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.067+0000 7fd741f91640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fd73c0c9e50 0x7fd73c0c3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:57020/0 (socket says 172.21.15.145:57020) 2024-04-22T07:03:47.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.067+0000 7fd740f8f640 1 -- 172.21.15.145:0/2459984215 learned_addr learned my addr 172.21.15.145:0/2459984215 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:47.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.067+0000 7fd741790640 1 --2- 172.21.15.145:0/2459984215 >> v2:172.21.15.92:3300/0 conn(0x7fd73c156220 0x7fd73c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.068+0000 7fd740f8f640 1 -- 172.21.15.145:0/2459984215 >> v2:172.21.15.92:3300/0 conn(0x7fd73c156220 msgr2=0x7fd73c070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:47.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.068+0000 7fd740f8f640 1 --2- 172.21.15.145:0/2459984215 >> v2:172.21.15.92:3300/0 conn(0x7fd73c156220 0x7fd73c070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.068+0000 7fd740f8f640 1 -- 172.21.15.145:0/2459984215 >> v1:172.21.15.145:6789/0 conn(0x7fd73c0c9e50 legacy=0x7fd73c0c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.068+0000 7fd740f8f640 1 -- 172.21.15.145:0/2459984215 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c0c36b0 con 0x7fd73c0c8df0 2024-04-22T07:03:47.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.068+0000 7fd741790640 1 --2- 172.21.15.145:0/2459984215 >> v2:172.21.15.92:3300/0 conn(0x7fd73c156220 0x7fd73c070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:03:47.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.068+0000 7fd740f8f640 1 --2- 172.21.15.145:0/2459984215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd73c0c8df0 0x7fd73c159510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0cbc40 tx=0x7fd72c02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52d4bb914d7b081f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:47.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.068+0000 7fd733fff640 1 -- 172.21.15.145:0/2459984215 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c034070 con 0x7fd73c0c8df0 2024-04-22T07:03:47.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.068+0000 7fd733fff640 1 -- 172.21.15.145:0/2459984215 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72c030860 con 0x7fd73c0c8df0 2024-04-22T07:03:47.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.069+0000 7fd733fff640 1 -- 172.21.15.145:0/2459984215 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c030c30 con 0x7fd73c0c8df0 2024-04-22T07:03:47.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.069+0000 7fd7439f4640 1 -- 172.21.15.145:0/2459984215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd73c0c8df0 msgr2=0x7fd73c159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.069+0000 7fd7439f4640 1 --2- 172.21.15.145:0/2459984215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd73c0c8df0 0x7fd73c159510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0cbc40 tx=0x7fd72c02b9a0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.069+0000 7fd7439f4640 1 -- 172.21.15.145:0/2459984215 shutdown_connections 2024-04-22T07:03:47.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.069+0000 7fd7439f4640 1 --2- 172.21.15.145:0/2459984215 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd73c0c8df0 0x7fd73c159510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.069+0000 7fd7439f4640 1 --2- 172.21.15.145:0/2459984215 >> v2:172.21.15.92:3300/0 conn(0x7fd73c156220 0x7fd73c070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.069+0000 7fd7439f4640 1 -- 172.21.15.145:0/2459984215 >> 172.21.15.145:0/2459984215 conn(0x7fd73c06e210 msgr2=0x7fd73c071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:47.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.070+0000 7fd7439f4640 1 -- 172.21.15.145:0/2459984215 shutdown_connections 2024-04-22T07:03:47.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.070+0000 7fd7439f4640 1 -- 172.21.15.145:0/2459984215 wait complete. 2024-04-22T07:03:47.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.071+0000 7fd7439f4640 1 Processor -- start 2024-04-22T07:03:47.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.071+0000 7fd7439f4640 1 -- start start 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.072+0000 7fd7439f4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd73c0c8df0 0x7fd73c144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.072+0000 7fd7439f4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd73c0c9e50 0x7fd73c1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.072+0000 7fd7439f4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd73c156220 0x7fd73c0b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd7439f4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c0720c0 con 0x7fd73c0c9e50 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd7439f4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c06fc70 con 0x7fd73c156220 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd7439f4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd73c156600 con 0x7fd73c0c8df0 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd741790640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd73c0c8df0 0x7fd73c144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd741790640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd73c0c8df0 0x7fd73c144690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:35240/0 (socket says 172.21.15.145:35240) 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd741790640 1 -- 172.21.15.145:0/76645971 learned_addr learned my addr 172.21.15.145:0/76645971 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd741f91640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd73c156220 0x7fd73c0b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd740f8f640 1 --2- 172.21.15.145:0/76645971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd73c0c9e50 0x7fd73c1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd741790640 1 -- 172.21.15.145:0/76645971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd73c156220 msgr2=0x7fd73c0b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd741790640 1 --2- 172.21.15.145:0/76645971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd73c156220 0x7fd73c0b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd741790640 1 -- 172.21.15.145:0/76645971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd73c0c9e50 msgr2=0x7fd73c1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd741790640 1 --2- 172.21.15.145:0/76645971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd73c0c9e50 0x7fd73c1568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd741790640 1 -- 172.21.15.145:0/76645971 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c148620 con 0x7fd73c0c8df0 2024-04-22T07:03:47.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd741f91640 1 --2- 172.21.15.145:0/76645971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd73c156220 0x7fd73c0b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:47.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd741790640 1 --2- 172.21.15.145:0/76645971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd73c0c8df0 0x7fd73c144690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd73800d800 tx=0x7fd73800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:47.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd731ffb640 1 -- 172.21.15.145:0/76645971 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738004030 con 0x7fd73c0c8df0 2024-04-22T07:03:47.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd731ffb640 1 -- 172.21.15.145:0/76645971 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7380041d0 con 0x7fd73c0c8df0 2024-04-22T07:03:47.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd7439f4640 1 -- 172.21.15.145:0/76645971 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c148c90 con 0x7fd73c0c8df0 2024-04-22T07:03:47.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.073+0000 7fd731ffb640 1 -- 172.21.15.145:0/76645971 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73800b6c0 con 0x7fd73c0c8df0 2024-04-22T07:03:47.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.074+0000 7fd7439f4640 1 -- 172.21.15.145:0/76645971 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c1f19f0 con 0x7fd73c0c8df0 2024-04-22T07:03:47.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.074+0000 7fd731ffb640 1 -- 172.21.15.145:0/76645971 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd73800b860 con 0x7fd73c0c8df0 2024-04-22T07:03:47.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.074+0000 7fd731ffb640 1 -- 172.21.15.145:0/76645971 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fd738003de0 con 0x7fd73c0c8df0 2024-04-22T07:03:47.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.075+0000 7fd7439f4640 1 -- 172.21.15.145:0/76645971 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73c1f17e0 con 0x7fd73c0c8df0 2024-04-22T07:03:47.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.077+0000 7fd731ffb640 1 -- 172.21.15.145:0/76645971 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd73800bb50 con 0x7fd73c0c8df0 2024-04-22T07:03:47.158 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:47.160+0000 7fdde9638040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T07:03:47.164 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.167+0000 7fd7439f4640 1 -- 172.21.15.145:0/76645971 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3edb4281-5113-4f74-930f-bf39cfae1d14", "id": 3} v 0) -- 0x7fd73c068b00 con 0x7fd73c0c8df0 2024-04-22T07:03:47.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.172+0000 7fd731ffb640 1 -- 172.21.15.145:0/76645971 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3edb4281-5113-4f74-930f-bf39cfae1d14", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fd738003610 con 0x7fd73c0c8df0 2024-04-22T07:03:47.170 INFO:teuthology.orchestra.run.smithi145.stdout:3 2024-04-22T07:03:47.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.173+0000 7fd7439f4640 1 -- 172.21.15.145:0/76645971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd73c0c8df0 msgr2=0x7fd73c144690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.173+0000 7fd7439f4640 1 --2- 172.21.15.145:0/76645971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd73c0c8df0 0x7fd73c144690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd73800d800 tx=0x7fd73800dcc0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.173+0000 7fd7439f4640 1 -- 172.21.15.145:0/76645971 shutdown_connections 2024-04-22T07:03:47.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.173+0000 7fd7439f4640 1 --2- 172.21.15.145:0/76645971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd73c156220 0x7fd73c0b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.173+0000 7fd7439f4640 1 --2- 172.21.15.145:0/76645971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd73c0c9e50 0x7fd73c1568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.173+0000 7fd7439f4640 1 --2- 172.21.15.145:0/76645971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd73c0c8df0 0x7fd73c144690 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.173+0000 7fd7439f4640 1 -- 172.21.15.145:0/76645971 >> 172.21.15.145:0/76645971 conn(0x7fd73c06e210 msgr2=0x7fd73c0ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:47.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.173+0000 7fd7439f4640 1 -- 172.21.15.145:0/76645971 shutdown_connections 2024-04-22T07:03:47.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.173+0000 7fd7439f4640 1 -- 172.21.15.145:0/76645971 wait complete. 2024-04-22T07:03:47.181 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 078f7b2e-f718-498e-82c2-907970d4497e 4 2024-04-22T07:03:47.246 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:47.248+0000 7fdde9638040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T07:03:47.268 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:47.270+0000 7efe53c41040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T07:03:47.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.281+0000 7f6d7bb52640 1 Processor -- start 2024-04-22T07:03:47.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.281+0000 7f6d7bb52640 1 -- start start 2024-04-22T07:03:47.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.282+0000 7f6d7bb52640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6d7414f5f0 0x7f6d740c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.282+0000 7f6d7bb52640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d740c7d40 0x7f6d7415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.282+0000 7f6d7bb52640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d740c7370 con 0x7f6d7415c4f0 2024-04-22T07:03:47.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.282+0000 7f6d7bb52640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f6d7406ce30 con 0x7f6d7414f5f0 2024-04-22T07:03:47.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.282+0000 7f6d7bb52640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6d7406d2c0 con 0x7f6d740c7d40 2024-04-22T07:03:47.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.282+0000 7f6d790ed640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d740c7d40 0x7f6d7415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.282+0000 7f6d790ed640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d740c7d40 0x7f6d7415bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:35256/0 (socket says 172.21.15.145:35256) 2024-04-22T07:03:47.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.282+0000 7f6d7a0ef640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f6d7415c4f0 0x7f6d74070a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:57034/0 (socket says 172.21.15.145:57034) 2024-04-22T07:03:47.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.282+0000 7f6d790ed640 1 -- 172.21.15.145:0/3632673119 learned_addr learned my addr 172.21.15.145:0/3632673119 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:47.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.282+0000 7f6d798ee640 1 --2- 172.21.15.145:0/3632673119 >> v2:172.21.15.92:3300/0 conn(0x7f6d7414f5f0 0x7f6d740c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.283+0000 7f6d790ed640 1 -- 172.21.15.145:0/3632673119 >> v2:172.21.15.92:3300/0 conn(0x7f6d7414f5f0 msgr2=0x7f6d740c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:47.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.283+0000 7f6d790ed640 1 --2- 172.21.15.145:0/3632673119 >> v2:172.21.15.92:3300/0 conn(0x7f6d7414f5f0 0x7f6d740c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.283+0000 7f6d790ed640 1 -- 172.21.15.145:0/3632673119 >> v1:172.21.15.145:6789/0 conn(0x7f6d7415c4f0 legacy=0x7f6d74070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.283+0000 7f6d790ed640 1 -- 172.21.15.145:0/3632673119 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d740c6db0 con 0x7f6d740c7d40 2024-04-22T07:03:47.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.283+0000 7f6d788ec640 1 -- 172.21.15.145:0/3632673119 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 126112042 0 0) 0x7f6d740c7370 con 0x7f6d7415c4f0 2024-04-22T07:03:47.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.283+0000 7f6d790ed640 1 --2- 172.21.15.145:0/3632673119 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d740c7d40 0x7f6d7415bfb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009870 tx=0x7f6d7002ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0ed37c9a8219321 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:47.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.283+0000 7f6d788ec640 1 -- 172.21.15.145:0/3632673119 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7002f840 con 0x7f6d740c7d40 2024-04-22T07:03:47.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.284+0000 7f6d788ec640 1 -- 172.21.15.145:0/3632673119 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d7002f9e0 con 0x7f6d740c7d40 2024-04-22T07:03:47.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.284+0000 7f6d788ec640 1 -- 172.21.15.145:0/3632673119 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7002fdb0 con 0x7f6d740c7d40 2024-04-22T07:03:47.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.284+0000 7f6d7bb52640 1 -- 172.21.15.145:0/3632673119 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d740c7d40 msgr2=0x7f6d7415bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.284+0000 7f6d7bb52640 1 --2- 172.21.15.145:0/3632673119 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d740c7d40 0x7f6d7415bfb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009870 tx=0x7f6d7002ba20 comp rx=0 tx=0).stop 2024-04-22T07:03:47.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.284+0000 7f6d7bb52640 1 -- 172.21.15.145:0/3632673119 shutdown_connections 2024-04-22T07:03:47.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.284+0000 7f6d7bb52640 1 --2- 172.21.15.145:0/3632673119 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d740c7d40 0x7f6d7415bfb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.284+0000 7f6d7bb52640 1 --2- 172.21.15.145:0/3632673119 >> v2:172.21.15.92:3300/0 conn(0x7f6d7414f5f0 0x7f6d740c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.284+0000 7f6d7bb52640 1 -- 172.21.15.145:0/3632673119 >> 172.21.15.145:0/3632673119 conn(0x7f6d740cd340 msgr2=0x7f6d740cd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:47.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.284+0000 7f6d7bb52640 1 -- 172.21.15.145:0/3632673119 shutdown_connections 2024-04-22T07:03:47.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.284+0000 7f6d7bb52640 1 -- 172.21.15.145:0/3632673119 wait complete. 2024-04-22T07:03:47.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.285+0000 7f6d7bb52640 1 Processor -- start 2024-04-22T07:03:47.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.285+0000 7f6d7bb52640 1 -- start start 2024-04-22T07:03:47.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7bb52640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d740c7d40 0x7f6d7415bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7bb52640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d7414f5f0 0x7f6d741503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7bb52640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d7415c4f0 0x7f6d74151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7bb52640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7406cac0 con 0x7f6d7414f5f0 2024-04-22T07:03:47.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7bb52640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7406c7e0 con 0x7f6d740c7d40 2024-04-22T07:03:47.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7bb52640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6d740c81b0 con 0x7f6d7415c4f0 2024-04-22T07:03:47.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7a0ef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d7415c4f0 0x7f6d74151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d798ee640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d740c7d40 0x7f6d7415bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7a0ef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d7415c4f0 0x7f6d74151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:35264/0 (socket says 172.21.15.145:35264) 2024-04-22T07:03:47.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d798ee640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d740c7d40 0x7f6d7415bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:34270/0 (socket says 172.21.15.145:34270) 2024-04-22T07:03:47.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7a0ef640 1 -- 172.21.15.145:0/44320 learned_addr learned my addr 172.21.15.145:0/44320 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:47.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d790ed640 1 --2- 172.21.15.145:0/44320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d7414f5f0 0x7f6d741503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7a0ef640 1 -- 172.21.15.145:0/44320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d740c7d40 msgr2=0x7f6d7415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7a0ef640 1 --2- 172.21.15.145:0/44320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d740c7d40 0x7f6d7415bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7a0ef640 1 -- 172.21.15.145:0/44320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d7414f5f0 msgr2=0x7f6d741503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7a0ef640 1 --2- 172.21.15.145:0/44320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d7414f5f0 0x7f6d741503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d7a0ef640 1 -- 172.21.15.145:0/44320 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7415c190 con 0x7f6d7415c4f0 2024-04-22T07:03:47.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.286+0000 7f6d798ee640 1 --2- 172.21.15.145:0/44320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d740c7d40 0x7f6d7415bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:47.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.287+0000 7f6d7a0ef640 1 --2- 172.21.15.145:0/44320 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d7415c4f0 0x7f6d74151d40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c7b0 tx=0x7f6d6c00cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:47.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.287+0000 7f6d6a7fc640 1 -- 172.21.15.145:0/44320 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c018070 con 0x7f6d7415c4f0 2024-04-22T07:03:47.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.287+0000 7f6d6a7fc640 1 -- 172.21.15.145:0/44320 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d6c00a5e0 con 0x7f6d7415c4f0 2024-04-22T07:03:47.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.287+0000 7f6d7bb52640 1 -- 172.21.15.145:0/44320 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d741f1b30 con 0x7f6d7415c4f0 2024-04-22T07:03:47.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.287+0000 7f6d6a7fc640 1 -- 172.21.15.145:0/44320 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c00ce60 con 0x7f6d7415c4f0 2024-04-22T07:03:47.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.287+0000 7f6d7bb52640 1 -- 172.21.15.145:0/44320 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d741f20f0 con 0x7f6d7415c4f0 2024-04-22T07:03:47.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.287+0000 7f6d6a7fc640 1 -- 172.21.15.145:0/44320 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6d6c01b070 con 0x7f6d7415c4f0 2024-04-22T07:03:47.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.287+0000 7f6d6a7fc640 1 -- 172.21.15.145:0/44320 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f6d6c0077b0 con 0x7f6d7415c4f0 2024-04-22T07:03:47.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.288+0000 7f6d7bb52640 1 -- 172.21.15.145:0/44320 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d74068b00 con 0x7f6d7415c4f0 2024-04-22T07:03:47.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.290+0000 7f6d6a7fc640 1 -- 172.21.15.145:0/44320 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6d6c0070d0 con 0x7f6d7415c4f0 2024-04-22T07:03:47.291 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:47.291+0000 7fdde9638040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T07:03:47.401 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:47.404+0000 7efe53c41040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T07:03:47.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.423+0000 7f6d7bb52640 1 -- 172.21.15.145:0/44320 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd new", "uuid": "078f7b2e-f718-498e-82c2-907970d4497e", "id": 4} v 0) -- 0x7f6d74070830 con 0x7f6d7415c4f0 2024-04-22T07:03:47.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.428+0000 7f6d6a7fc640 1 -- 172.21.15.145:0/44320 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "078f7b2e-f718-498e-82c2-907970d4497e", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f6d6c007380 con 0x7f6d7415c4f0 2024-04-22T07:03:47.426 INFO:teuthology.orchestra.run.smithi145.stdout:4 2024-04-22T07:03:47.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.429+0000 7f6d7bb52640 1 -- 172.21.15.145:0/44320 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d7415c4f0 msgr2=0x7f6d74151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.429+0000 7f6d7bb52640 1 --2- 172.21.15.145:0/44320 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d7415c4f0 0x7f6d74151d40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c7b0 tx=0x7f6d6c00cc70 comp rx=0 tx=0).stop 2024-04-22T07:03:47.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.429+0000 7f6d7bb52640 1 -- 172.21.15.145:0/44320 shutdown_connections 2024-04-22T07:03:47.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.429+0000 7f6d7bb52640 1 --2- 172.21.15.145:0/44320 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d7415c4f0 0x7f6d74151d40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.429+0000 7f6d7bb52640 1 --2- 172.21.15.145:0/44320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d7414f5f0 0x7f6d741503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.429+0000 7f6d7bb52640 1 --2- 172.21.15.145:0/44320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d740c7d40 0x7f6d7415bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.429+0000 7f6d7bb52640 1 -- 172.21.15.145:0/44320 >> 172.21.15.145:0/44320 conn(0x7f6d740cd340 msgr2=0x7f6d74148db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:47.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.429+0000 7f6d7bb52640 1 -- 172.21.15.145:0/44320 shutdown_connections 2024-04-22T07:03:47.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.429+0000 7f6d7bb52640 1 -- 172.21.15.145:0/44320 wait complete. 2024-04-22T07:03:47.437 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new fe0e1c02-ccf8-441a-8ed6-afcff9db1565 5 2024-04-22T07:03:47.448 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:47.451+0000 7efe53c41040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T07:03:47.456 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:47.458+0000 7fdde9638040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T07:03:47.504 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:47.506+0000 7fdde9638040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T07:03:47.534 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.537+0000 7f7385f75640 1 Processor -- start 2024-04-22T07:03:47.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.537+0000 7f7385f75640 1 -- start start 2024-04-22T07:03:47.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.537+0000 7f7385f75640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73800c6cc0 0x7f738006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.537+0000 7f7385f75640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 0x7f7380158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.537+0000 7f7385f75640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f73800ca990 0x7f7380152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.537+0000 7f7385f75640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f73800c7d40 con 0x7f73800c6cc0 2024-04-22T07:03:47.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.537+0000 7f7385f75640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f73800c8000 con 0x7f73800ca990 2024-04-22T07:03:47.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.537+0000 7f7385f75640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f73800c82e0 con 0x7f73800c70a0 2024-04-22T07:03:47.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.537+0000 7f737effd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 0x7f7380158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.538+0000 7f737ffff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f73800ca990 0x7f7380152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.538+0000 7f737effd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 0x7f7380158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:35266/0 (socket says 172.21.15.145:35266) 2024-04-22T07:03:47.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.538+0000 7f737effd640 1 -- 172.21.15.145:0/2306149015 learned_addr learned my addr 172.21.15.145:0/2306149015 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:47.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.538+0000 7f737f7fe640 1 --2- 172.21.15.145:0/2306149015 >> v2:172.21.15.92:3300/0 conn(0x7f73800c6cc0 0x7f738006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.538+0000 7f737effd640 1 -- 172.21.15.145:0/2306149015 >> v2:172.21.15.145:3300/0 conn(0x7f73800ca990 msgr2=0x7f7380152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:47.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.538+0000 7f737effd640 1 --2- 172.21.15.145:0/2306149015 >> v2:172.21.15.145:3300/0 conn(0x7f73800ca990 0x7f7380152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.538+0000 7f737effd640 1 -- 172.21.15.145:0/2306149015 >> v2:172.21.15.92:3300/0 conn(0x7f73800c6cc0 msgr2=0x7f738006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:47.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.538+0000 7f737effd640 1 --2- 172.21.15.145:0/2306149015 >> v2:172.21.15.92:3300/0 conn(0x7f73800c6cc0 0x7f738006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.538+0000 7f737effd640 1 -- 172.21.15.145:0/2306149015 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73800678c0 con 0x7f73800c70a0 2024-04-22T07:03:47.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.538+0000 7f737ffff640 1 --2- 172.21.15.145:0/2306149015 >> v2:172.21.15.145:3300/0 conn(0x7f73800ca990 0x7f7380152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:47.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f737effd640 1 --2- 172.21.15.145:0/2306149015 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 0x7f7380158b80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f73801567b0 tx=0x7f737402b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9ae5eb686e4728 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:47.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f737e7fc640 1 -- 172.21.15.145:0/2306149015 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374035020 con 0x7f73800c70a0 2024-04-22T07:03:47.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f737e7fc640 1 -- 172.21.15.145:0/2306149015 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7374031870 con 0x7f73800c70a0 2024-04-22T07:03:47.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f737e7fc640 1 -- 172.21.15.145:0/2306149015 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374031c40 con 0x7f73800c70a0 2024-04-22T07:03:47.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f7385f75640 1 -- 172.21.15.145:0/2306149015 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 msgr2=0x7f7380158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f7385f75640 1 --2- 172.21.15.145:0/2306149015 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 0x7f7380158b80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f73801567b0 tx=0x7f737402b910 comp rx=0 tx=0).stop 2024-04-22T07:03:47.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f7385f75640 1 -- 172.21.15.145:0/2306149015 shutdown_connections 2024-04-22T07:03:47.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f7385f75640 1 --2- 172.21.15.145:0/2306149015 >> v2:172.21.15.145:3300/0 conn(0x7f73800ca990 0x7f7380152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f7385f75640 1 --2- 172.21.15.145:0/2306149015 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 0x7f7380158b80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f7385f75640 1 --2- 172.21.15.145:0/2306149015 >> v2:172.21.15.92:3300/0 conn(0x7f73800c6cc0 0x7f738006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f7385f75640 1 -- 172.21.15.145:0/2306149015 >> 172.21.15.145:0/2306149015 conn(0x7f7380072ee0 msgr2=0x7f738006e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:47.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f7385f75640 1 -- 172.21.15.145:0/2306149015 shutdown_connections 2024-04-22T07:03:47.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.539+0000 7f7385f75640 1 -- 172.21.15.145:0/2306149015 wait complete. 2024-04-22T07:03:47.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.541+0000 7f7385f75640 1 Processor -- start 2024-04-22T07:03:47.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.541+0000 7f7385f75640 1 -- start start 2024-04-22T07:03:47.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.541+0000 7f7385f75640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73800c6cc0 0x7f73801592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.541+0000 7f7385f75640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 0x7f73801dede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.541+0000 7f7385f75640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73800ca990 0x7f73800cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.541+0000 7f7385f75640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f73800c9c60 con 0x7f73800c6cc0 2024-04-22T07:03:47.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.541+0000 7f7385f75640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7380067e80 con 0x7f73800ca990 2024-04-22T07:03:47.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.541+0000 7f7385f75640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f73800c9de0 con 0x7f73800c70a0 2024-04-22T07:03:47.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.541+0000 7f737effd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 0x7f73801dede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.541+0000 7f737ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73800ca990 0x7f73800cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f737effd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 0x7f73801dede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:35272/0 (socket says 172.21.15.145:35272) 2024-04-22T07:03:47.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f737effd640 1 -- 172.21.15.145:0/2667212386 learned_addr learned my addr 172.21.15.145:0/2667212386 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:47.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f737ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73800ca990 0x7f73800cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:34292/0 (socket says 172.21.15.145:34292) 2024-04-22T07:03:47.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f737f7fe640 1 --2- 172.21.15.145:0/2667212386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73800c6cc0 0x7f73801592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f737ffff640 1 -- 172.21.15.145:0/2667212386 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 msgr2=0x7f73801dede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f737ffff640 1 --2- 172.21.15.145:0/2667212386 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 0x7f73801dede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f737ffff640 1 -- 172.21.15.145:0/2667212386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73800c6cc0 msgr2=0x7f73801592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f737ffff640 1 --2- 172.21.15.145:0/2667212386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73800c6cc0 0x7f73801592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f737ffff640 1 -- 172.21.15.145:0/2667212386 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73801598e0 con 0x7f73800ca990 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f737ffff640 1 --2- 172.21.15.145:0/2667212386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73800ca990 0x7f73800cd9a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f737800efc0 tx=0x7f737800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f7367fff640 1 -- 172.21.15.145:0/2667212386 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378018020 con 0x7f73800ca990 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f7367fff640 1 -- 172.21.15.145:0/2667212386 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7378014480 con 0x7f73800ca990 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f7385f75640 1 -- 172.21.15.145:0/2667212386 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7380159bc0 con 0x7f73800ca990 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f7367fff640 1 -- 172.21.15.145:0/2667212386 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378014870 con 0x7f73800ca990 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f7385f75640 1 -- 172.21.15.145:0/2667212386 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73801f0ef0 con 0x7f73800ca990 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f7367fff640 1 -- 172.21.15.145:0/2667212386 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f737801b020 con 0x7f73800ca990 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.542+0000 7f7367fff640 1 -- 172.21.15.145:0/2667212386 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f737801dcc0 con 0x7f73800ca990 2024-04-22T07:03:47.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.543+0000 7f7385f75640 1 -- 172.21.15.145:0/2667212386 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7380068b00 con 0x7f73800ca990 2024-04-22T07:03:47.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.547+0000 7f7367fff640 1 -- 172.21.15.145:0/2667212386 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7378022020 con 0x7f73800ca990 2024-04-22T07:03:47.556 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:47.558+0000 7fdde9638040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T07:03:47.617 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.620+0000 7f7385f75640 1 -- 172.21.15.145:0/2667212386 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fe0e1c02-ccf8-441a-8ed6-afcff9db1565", "id": 5} v 0) -- 0x7f738014cd10 con 0x7f73800ca990 2024-04-22T07:03:47.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.625+0000 7f7367fff640 1 -- 172.21.15.145:0/2667212386 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fe0e1c02-ccf8-441a-8ed6-afcff9db1565", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f7378010040 con 0x7f73800ca990 2024-04-22T07:03:47.623 INFO:teuthology.orchestra.run.smithi145.stdout:5 2024-04-22T07:03:47.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.626+0000 7f7385f75640 1 -- 172.21.15.145:0/2667212386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73800ca990 msgr2=0x7f73800cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.626+0000 7f7385f75640 1 --2- 172.21.15.145:0/2667212386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73800ca990 0x7f73800cd9a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f737800efc0 tx=0x7f737800c490 comp rx=0 tx=0).stop 2024-04-22T07:03:47.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.626+0000 7f7385f75640 1 -- 172.21.15.145:0/2667212386 shutdown_connections 2024-04-22T07:03:47.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.626+0000 7f7385f75640 1 --2- 172.21.15.145:0/2667212386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73800ca990 0x7f73800cd9a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.626+0000 7f7385f75640 1 --2- 172.21.15.145:0/2667212386 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73800c70a0 0x7f73801dede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.626+0000 7f7385f75640 1 --2- 172.21.15.145:0/2667212386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73800c6cc0 0x7f73801592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.626+0000 7f7385f75640 1 -- 172.21.15.145:0/2667212386 >> 172.21.15.145:0/2667212386 conn(0x7f7380072ee0 msgr2=0x7f738014fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:47.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.626+0000 7f7385f75640 1 -- 172.21.15.145:0/2667212386 shutdown_connections 2024-04-22T07:03:47.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.626+0000 7f7385f75640 1 -- 172.21.15.145:0/2667212386 wait complete. 2024-04-22T07:03:47.625 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:47.627+0000 7efe53c41040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T07:03:47.634 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 7c9dc8bd-f1c0-4005-956d-928a1ce7636c 6 2024-04-22T07:03:47.668 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:47.670+0000 7fdde9638040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:03:47.684 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:47.687+0000 7efe53c41040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T07:03:47.740 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:47.742+0000 7efe53c41040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T07:03:47.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.775+0000 7f8e48366640 1 Processor -- start 2024-04-22T07:03:47.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.775+0000 7f8e48366640 1 -- start start 2024-04-22T07:03:47.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e48366640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8e40070cb0 0x7f8e400c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e48366640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8e40071090 0x7f8e401517e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e48366640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e4015af60 con 0x7f8e400c7ca0 2024-04-22T07:03:47.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e48366640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4006fc40 con 0x7f8e40071090 2024-04-22T07:03:47.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e48366640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4006ff20 con 0x7f8e40070cb0 2024-04-22T07:03:47.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e46102640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8e40070cb0 0x7f8e400c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e46102640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8e40070cb0 0x7f8e400c47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:34300/0 (socket says 172.21.15.145:34300) 2024-04-22T07:03:47.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e46102640 1 -- 172.21.15.145:0/4070810569 learned_addr learned my addr 172.21.15.145:0/4070810569 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:47.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e45901640 1 --2- 172.21.15.145:0/4070810569 >> v2:172.21.15.92:3300/0 conn(0x7f8e40071090 0x7f8e401517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e46102640 1 -- 172.21.15.145:0/4070810569 >> v2:172.21.15.92:3300/0 conn(0x7f8e40071090 msgr2=0x7f8e401517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:47.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e46102640 1 --2- 172.21.15.145:0/4070810569 >> v2:172.21.15.92:3300/0 conn(0x7f8e40071090 0x7f8e401517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e46102640 1 -- 172.21.15.145:0/4070810569 >> v1:172.21.15.92:6789/0 conn(0x7f8e400c7ca0 legacy=0x7f8e4015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.776+0000 7f8e46102640 1 -- 172.21.15.145:0/4070810569 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e40070200 con 0x7f8e40070cb0 2024-04-22T07:03:47.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.777+0000 7f8e46102640 1 --2- 172.21.15.145:0/4070810569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e40070cb0 0x7f8e400c47a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8e4006d4c0 tx=0x7f8e3c02e170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=923fd55af0a39edd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:47.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.777+0000 7f8e45100640 1 -- 172.21.15.145:0/4070810569 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c03b070 con 0x7f8e40070cb0 2024-04-22T07:03:47.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.777+0000 7f8e45100640 1 -- 172.21.15.145:0/4070810569 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e3c036440 con 0x7f8e40070cb0 2024-04-22T07:03:47.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.777+0000 7f8e45100640 1 -- 172.21.15.145:0/4070810569 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c036810 con 0x7f8e40070cb0 2024-04-22T07:03:47.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.778+0000 7f8e48366640 1 -- 172.21.15.145:0/4070810569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e40070cb0 msgr2=0x7f8e400c47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.778+0000 7f8e48366640 1 --2- 172.21.15.145:0/4070810569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e40070cb0 0x7f8e400c47a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8e4006d4c0 tx=0x7f8e3c02e170 comp rx=0 tx=0).stop 2024-04-22T07:03:47.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.778+0000 7f8e48366640 1 -- 172.21.15.145:0/4070810569 shutdown_connections 2024-04-22T07:03:47.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.778+0000 7f8e48366640 1 --2- 172.21.15.145:0/4070810569 >> v2:172.21.15.92:3300/0 conn(0x7f8e40071090 0x7f8e401517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.778+0000 7f8e48366640 1 --2- 172.21.15.145:0/4070810569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e40070cb0 0x7f8e400c47a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.778+0000 7f8e48366640 1 -- 172.21.15.145:0/4070810569 >> 172.21.15.145:0/4070810569 conn(0x7f8e400cd2a0 msgr2=0x7f8e400cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:47.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.778+0000 7f8e48366640 1 -- 172.21.15.145:0/4070810569 shutdown_connections 2024-04-22T07:03:47.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.778+0000 7f8e48366640 1 -- 172.21.15.145:0/4070810569 wait complete. 2024-04-22T07:03:47.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.779+0000 7f8e48366640 1 Processor -- start 2024-04-22T07:03:47.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.779+0000 7f8e48366640 1 -- start start 2024-04-22T07:03:47.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e48366640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e40070cb0 0x7f8e40159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e48366640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e40071090 0x7f8e400cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e48366640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e400c7ca0 0x7f8e40157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e48366640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e400c4ce0 con 0x7f8e40070cb0 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e48366640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e400707c0 con 0x7f8e400c7ca0 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e48366640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4006d140 con 0x7f8e40071090 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e46903640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e400c7ca0 0x7f8e40157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e45901640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e40071090 0x7f8e400cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e46903640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e400c7ca0 0x7f8e40157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:34304/0 (socket says 172.21.15.145:34304) 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e45901640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e40071090 0x7f8e400cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:35274/0 (socket says 172.21.15.145:35274) 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e46903640 1 -- 172.21.15.145:0/681558865 learned_addr learned my addr 172.21.15.145:0/681558865 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e46102640 1 --2- 172.21.15.145:0/681558865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e40070cb0 0x7f8e40159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e46903640 1 -- 172.21.15.145:0/681558865 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e40071090 msgr2=0x7f8e400cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e46903640 1 --2- 172.21.15.145:0/681558865 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e40071090 0x7f8e400cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e46903640 1 -- 172.21.15.145:0/681558865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e40070cb0 msgr2=0x7f8e40159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e46903640 1 --2- 172.21.15.145:0/681558865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e40070cb0 0x7f8e40159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e46903640 1 -- 172.21.15.145:0/681558865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4015a210 con 0x7f8e400c7ca0 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e45901640 1 --2- 172.21.15.145:0/681558865 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e40071090 0x7f8e400cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e46903640 1 --2- 172.21.15.145:0/681558865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e400c7ca0 0x7f8e40157b20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800c760 tx=0x7f8e3800cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e36ffd640 1 -- 172.21.15.145:0/681558865 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38003b90 con 0x7f8e400c7ca0 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e36ffd640 1 -- 172.21.15.145:0/681558865 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e38003d30 con 0x7f8e400c7ca0 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e36ffd640 1 -- 172.21.15.145:0/681558865 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3800aa90 con 0x7f8e400c7ca0 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.780+0000 7f8e48366640 1 -- 172.21.15.145:0/681558865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e40158c30 con 0x7f8e400c7ca0 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.781+0000 7f8e48366640 1 -- 172.21.15.145:0/681558865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e401591f0 con 0x7f8e400c7ca0 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.781+0000 7f8e36ffd640 1 -- 172.21.15.145:0/681558865 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8e3800ac30 con 0x7f8e400c7ca0 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.781+0000 7f8e36ffd640 1 -- 172.21.15.145:0/681558865 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f8e38012bf0 con 0x7f8e400c7ca0 2024-04-22T07:03:47.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.781+0000 7f8e48366640 1 -- 172.21.15.145:0/681558865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e400c6e40 con 0x7f8e400c7ca0 2024-04-22T07:03:47.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.785+0000 7f8e36ffd640 1 -- 172.21.15.145:0/681558865 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8e38012600 con 0x7f8e400c7ca0 2024-04-22T07:03:47.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.858+0000 7f8e48366640 1 -- 172.21.15.145:0/681558865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7c9dc8bd-f1c0-4005-956d-928a1ce7636c", "id": 6} v 0) -- 0x7f8e4014b560 con 0x7f8e400c7ca0 2024-04-22T07:03:47.860 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:47.862+0000 7efe53c41040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:03:47.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.866+0000 7f8e36ffd640 1 -- 172.21.15.145:0/681558865 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7c9dc8bd-f1c0-4005-956d-928a1ce7636c", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f8e380127e0 con 0x7f8e400c7ca0 2024-04-22T07:03:47.863 INFO:teuthology.orchestra.run.smithi145.stdout:6 2024-04-22T07:03:47.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.867+0000 7f8e48366640 1 -- 172.21.15.145:0/681558865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e400c7ca0 msgr2=0x7f8e40157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.867+0000 7f8e48366640 1 --2- 172.21.15.145:0/681558865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e400c7ca0 0x7f8e40157b20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800c760 tx=0x7f8e3800cc20 comp rx=0 tx=0).stop 2024-04-22T07:03:47.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.867+0000 7f8e48366640 1 -- 172.21.15.145:0/681558865 shutdown_connections 2024-04-22T07:03:47.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.867+0000 7f8e48366640 1 --2- 172.21.15.145:0/681558865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e400c7ca0 0x7f8e40157b20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.867+0000 7f8e48366640 1 --2- 172.21.15.145:0/681558865 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e40071090 0x7f8e400cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.867+0000 7f8e48366640 1 --2- 172.21.15.145:0/681558865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e40070cb0 0x7f8e40159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.867+0000 7f8e48366640 1 -- 172.21.15.145:0/681558865 >> 172.21.15.145:0/681558865 conn(0x7f8e400cd2a0 msgr2=0x7f8e4006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:47.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.867+0000 7f8e48366640 1 -- 172.21.15.145:0/681558865 shutdown_connections 2024-04-22T07:03:47.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:47.867+0000 7f8e48366640 1 -- 172.21.15.145:0/681558865 wait complete. 2024-04-22T07:03:47.875 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 51065971-9bc8-4005-baf2-39a4506048a4 7 2024-04-22T07:03:48.013 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.016+0000 7f3d2e031640 1 Processor -- start 2024-04-22T07:03:48.013 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.016+0000 7f3d2e031640 1 -- start start 2024-04-22T07:03:48.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.016+0000 7f3d2e031640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d28070cb0 0x7f3d280c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:48.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.016+0000 7f3d2e031640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3d28071090 0x7f3d281517e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:48.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.016+0000 7f3d2e031640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d2815af60 con 0x7f3d280c7ca0 2024-04-22T07:03:48.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.016+0000 7f3d2e031640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f3d2806fc40 con 0x7f3d28070cb0 2024-04-22T07:03:48.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.016+0000 7f3d2e031640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3d2806ff20 con 0x7f3d28071090 2024-04-22T07:03:48.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.016+0000 7f3d26ffd640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3d28071090 0x7f3d281517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:48.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.017+0000 7f3d26ffd640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3d28071090 0x7f3d281517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:34306/0 (socket says 172.21.15.145:34306) 2024-04-22T07:03:48.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.017+0000 7f3d26ffd640 1 -- 172.21.15.145:0/1247099382 learned_addr learned my addr 172.21.15.145:0/1247099382 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:48.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.017+0000 7f3d277fe640 1 --2- 172.21.15.145:0/1247099382 >> v2:172.21.15.92:3300/0 conn(0x7f3d28070cb0 0x7f3d280c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:48.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.017+0000 7f3d26ffd640 1 -- 172.21.15.145:0/1247099382 >> v2:172.21.15.92:3300/0 conn(0x7f3d28070cb0 msgr2=0x7f3d280c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:48.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.017+0000 7f3d26ffd640 1 --2- 172.21.15.145:0/1247099382 >> v2:172.21.15.92:3300/0 conn(0x7f3d28070cb0 0x7f3d280c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.017+0000 7f3d26ffd640 1 -- 172.21.15.145:0/1247099382 >> v1:172.21.15.92:6789/0 conn(0x7f3d280c7ca0 legacy=0x7f3d2815b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.017+0000 7f3d26ffd640 1 -- 172.21.15.145:0/1247099382 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d28070200 con 0x7f3d28071090 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.017+0000 7f3d277fe640 1 --2- 172.21.15.145:0/1247099382 >> v2:172.21.15.92:3300/0 conn(0x7f3d28070cb0 0x7f3d280c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.017+0000 7f3d26ffd640 1 --2- 172.21.15.145:0/1247099382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d28071090 0x7f3d281517e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3d28072100 tx=0x7f3d1802e170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1cfa1c185cd799a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.018+0000 7f3d267fc640 1 -- 172.21.15.145:0/1247099382 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1803b070 con 0x7f3d28071090 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.018+0000 7f3d267fc640 1 -- 172.21.15.145:0/1247099382 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d18036440 con 0x7f3d28071090 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.018+0000 7f3d267fc640 1 -- 172.21.15.145:0/1247099382 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18036810 con 0x7f3d28071090 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.018+0000 7f3d2e031640 1 -- 172.21.15.145:0/1247099382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d28071090 msgr2=0x7f3d281517e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.018+0000 7f3d2e031640 1 --2- 172.21.15.145:0/1247099382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d28071090 0x7f3d281517e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3d28072100 tx=0x7f3d1802e170 comp rx=0 tx=0).stop 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.018+0000 7f3d2e031640 1 -- 172.21.15.145:0/1247099382 shutdown_connections 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.018+0000 7f3d2e031640 1 --2- 172.21.15.145:0/1247099382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d28071090 0x7f3d281517e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.018+0000 7f3d2e031640 1 --2- 172.21.15.145:0/1247099382 >> v2:172.21.15.92:3300/0 conn(0x7f3d28070cb0 0x7f3d280c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.018+0000 7f3d2e031640 1 -- 172.21.15.145:0/1247099382 >> 172.21.15.145:0/1247099382 conn(0x7f3d280cd2a0 msgr2=0x7f3d280cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:48.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.018+0000 7f3d2e031640 1 -- 172.21.15.145:0/1247099382 shutdown_connections 2024-04-22T07:03:48.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.018+0000 7f3d2e031640 1 -- 172.21.15.145:0/1247099382 wait complete. 2024-04-22T07:03:48.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.019+0000 7f3d2e031640 1 Processor -- start 2024-04-22T07:03:48.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.020+0000 7f3d2e031640 1 -- start start 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.020+0000 7f3d2e031640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d28070cb0 0x7f3d280cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.020+0000 7f3d2e031640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d28071090 0x7f3d28145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.020+0000 7f3d2e031640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d280c7ca0 0x7f3d281ec650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.020+0000 7f3d2e031640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3d280c4ce0 con 0x7f3d28071090 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.020+0000 7f3d2e031640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3d280707c0 con 0x7f3d28070cb0 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.020+0000 7f3d2e031640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3d2806d140 con 0x7f3d280c7ca0 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.020+0000 7f3d277fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d28070cb0 0x7f3d280cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.020+0000 7f3d27fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d280c7ca0 0x7f3d281ec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.020+0000 7f3d277fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d28070cb0 0x7f3d280cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:34314/0 (socket says 172.21.15.145:34314) 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.020+0000 7f3d277fe640 1 -- 172.21.15.145:0/1170140050 learned_addr learned my addr 172.21.15.145:0/1170140050 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.020+0000 7f3d26ffd640 1 --2- 172.21.15.145:0/1170140050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d28071090 0x7f3d28145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d27fff640 1 -- 172.21.15.145:0/1170140050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d28070cb0 msgr2=0x7f3d280cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:48.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d27fff640 1 --2- 172.21.15.145:0/1170140050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d28070cb0 0x7f3d280cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d27fff640 1 -- 172.21.15.145:0/1170140050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d28071090 msgr2=0x7f3d28145c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d27fff640 1 --2- 172.21.15.145:0/1170140050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d28071090 0x7f3d28145c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d27fff640 1 -- 172.21.15.145:0/1170140050 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d281594a0 con 0x7f3d280c7ca0 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d277fe640 1 --2- 172.21.15.145:0/1170140050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d28070cb0 0x7f3d280cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d27fff640 1 --2- 172.21.15.145:0/1170140050 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d280c7ca0 0x7f3d281ec650 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000c730 tx=0x7f3d2000cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d0ffff640 1 -- 172.21.15.145:0/1170140050 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20003b90 con 0x7f3d280c7ca0 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d0ffff640 1 -- 172.21.15.145:0/1170140050 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d20003d50 con 0x7f3d280c7ca0 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d0ffff640 1 -- 172.21.15.145:0/1170140050 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20005310 con 0x7f3d280c7ca0 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d2e031640 1 -- 172.21.15.145:0/1170140050 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d281efd40 con 0x7f3d280c7ca0 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d2e031640 1 -- 172.21.15.145:0/1170140050 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281f0000 con 0x7f3d280c7ca0 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d0ffff640 1 -- 172.21.15.145:0/1170140050 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3d200056c0 con 0x7f3d280c7ca0 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.021+0000 7f3d0ffff640 1 -- 172.21.15.145:0/1170140050 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f3d20012d00 con 0x7f3d280c7ca0 2024-04-22T07:03:48.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.022+0000 7f3d2e031640 1 -- 172.21.15.145:0/1170140050 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d28068b00 con 0x7f3d280c7ca0 2024-04-22T07:03:48.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.024+0000 7f3d0ffff640 1 -- 172.21.15.145:0/1170140050 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3d281594a0 con 0x7f3d280c7ca0 2024-04-22T07:03:48.050 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:48.052+0000 7fdde9638040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T07:03:48.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.096+0000 7f3d2e031640 1 -- 172.21.15.145:0/1170140050 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd new", "uuid": "51065971-9bc8-4005-baf2-39a4506048a4", "id": 7} v 0) -- 0x7f3d2806cde0 con 0x7f3d280c7ca0 2024-04-22T07:03:48.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.104+0000 7f3d0ffff640 1 -- 172.21.15.145:0/1170140050 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "51065971-9bc8-4005-baf2-39a4506048a4", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f3d200126a0 con 0x7f3d280c7ca0 2024-04-22T07:03:48.102 INFO:teuthology.orchestra.run.smithi145.stdout:7 2024-04-22T07:03:48.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.105+0000 7f3d2e031640 1 -- 172.21.15.145:0/1170140050 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d280c7ca0 msgr2=0x7f3d281ec650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:48.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.105+0000 7f3d2e031640 1 --2- 172.21.15.145:0/1170140050 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d280c7ca0 0x7f3d281ec650 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000c730 tx=0x7f3d2000cbf0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.105+0000 7f3d2e031640 1 -- 172.21.15.145:0/1170140050 shutdown_connections 2024-04-22T07:03:48.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.105+0000 7f3d2e031640 1 --2- 172.21.15.145:0/1170140050 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d280c7ca0 0x7f3d281ec650 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.105+0000 7f3d2e031640 1 --2- 172.21.15.145:0/1170140050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d28071090 0x7f3d28145c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.105+0000 7f3d2e031640 1 --2- 172.21.15.145:0/1170140050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d28070cb0 0x7f3d280cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.105+0000 7f3d2e031640 1 -- 172.21.15.145:0/1170140050 >> 172.21.15.145:0/1170140050 conn(0x7f3d280cd2a0 msgr2=0x7f3d281515b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:48.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.105+0000 7f3d2e031640 1 -- 172.21.15.145:0/1170140050 shutdown_connections 2024-04-22T07:03:48.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-22T07:03:48.106+0000 7f3d2e031640 1 -- 172.21.15.145:0/1170140050 wait complete. 2024-04-22T07:03:48.114 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-22T07:03:48.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-22T07:03:48.116 INFO:tasks.ceph.osd.0:Started 2024-04-22T07:03:48.116 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-22T07:03:48.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-22T07:03:48.121 INFO:tasks.ceph.osd.1:Started 2024-04-22T07:03:48.121 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-22T07:03:48.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-22T07:03:48.123 INFO:tasks.ceph.osd.2:Started 2024-04-22T07:03:48.123 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-22T07:03:48.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-22T07:03:48.125 INFO:tasks.ceph.osd.3:Started 2024-04-22T07:03:48.125 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-22T07:03:48.125 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-22T07:03:48.127 INFO:tasks.ceph.osd.4:Started 2024-04-22T07:03:48.127 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-22T07:03:48.128 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-22T07:03:48.129 INFO:tasks.ceph.osd.5:Started 2024-04-22T07:03:48.130 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-22T07:03:48.130 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-22T07:03:48.132 INFO:tasks.ceph.osd.6:Started 2024-04-22T07:03:48.132 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-22T07:03:48.132 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-22T07:03:48.134 INFO:tasks.ceph.osd.7:Started 2024-04-22T07:03:48.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:03:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.252+0000 7f7d46a55640 1 Processor -- start 2024-04-22T07:03:48.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.252+0000 7f7d46a55640 1 -- start start 2024-04-22T07:03:48.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d46a55640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d4014def0 0x7f7d400c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:48.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d46a55640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d400c8790 0x7f7d400c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:48.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d46a55640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d400cd340 con 0x7f7d40071ce0 2024-04-22T07:03:48.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d46a55640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f7d4014e2d0 con 0x7f7d400c8790 2024-04-22T07:03:48.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d46a55640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7d40157a30 con 0x7f7d4014def0 2024-04-22T07:03:48.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d44ff2640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d400c8790 0x7f7d400c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d3f7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7d40071ce0 0x7f7d400c7ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55762/0 (socket says 172.21.15.92:55762) 2024-04-22T07:03:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d3f7fe640 1 -- 172.21.15.92:0/2332527050 learned_addr learned my addr 172.21.15.92:0/2332527050 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:03:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d3ffff640 1 --2- 172.21.15.92:0/2332527050 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d4014def0 0x7f7d400c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d44ff2640 1 -- 172.21.15.92:0/2332527050 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d4014def0 msgr2=0x7f7d400c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d44ff2640 1 --2- 172.21.15.92:0/2332527050 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d4014def0 0x7f7d400c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d44ff2640 1 -- 172.21.15.92:0/2332527050 >> v1:172.21.15.92:6789/0 conn(0x7f7d40071ce0 legacy=0x7f7d400c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d44ff2640 1 -- 172.21.15.92:0/2332527050 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d40157fb0 con 0x7f7d400c8790 2024-04-22T07:03:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.253+0000 7f7d3ffff640 1 --2- 172.21.15.92:0/2332527050 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d4014def0 0x7f7d400c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:03:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.254+0000 7f7d44ff2640 1 --2- 172.21.15.92:0/2332527050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d400c8790 0x7f7d400c9470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800bfa0 tx=0x7f7d3802e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48833b021d22658e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.254+0000 7f7d3effd640 1 -- 172.21.15.92:0/2332527050 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38034070 con 0x7f7d400c8790 2024-04-22T07:03:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.254+0000 7f7d3effd640 1 -- 172.21.15.92:0/2332527050 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d3802edc0 con 0x7f7d400c8790 2024-04-22T07:03:48.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.254+0000 7f7d3effd640 1 -- 172.21.15.92:0/2332527050 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38038620 con 0x7f7d400c8790 2024-04-22T07:03:48.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.255+0000 7f7d46a55640 1 -- 172.21.15.92:0/2332527050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d400c8790 msgr2=0x7f7d400c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:48.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.255+0000 7f7d46a55640 1 --2- 172.21.15.92:0/2332527050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d400c8790 0x7f7d400c9470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800bfa0 tx=0x7f7d3802e010 comp rx=0 tx=0).stop 2024-04-22T07:03:48.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.255+0000 7f7d46a55640 1 -- 172.21.15.92:0/2332527050 shutdown_connections 2024-04-22T07:03:48.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.255+0000 7f7d46a55640 1 --2- 172.21.15.92:0/2332527050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d400c8790 0x7f7d400c9470 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.255+0000 7f7d46a55640 1 --2- 172.21.15.92:0/2332527050 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d4014def0 0x7f7d400c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.255+0000 7f7d46a55640 1 -- 172.21.15.92:0/2332527050 >> 172.21.15.92:0/2332527050 conn(0x7f7d4006f8d0 msgr2=0x7f7d4006fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:48.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.255+0000 7f7d46a55640 1 -- 172.21.15.92:0/2332527050 shutdown_connections 2024-04-22T07:03:48.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.255+0000 7f7d46a55640 1 -- 172.21.15.92:0/2332527050 wait complete. 2024-04-22T07:03:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.256+0000 7f7d46a55640 1 Processor -- start 2024-04-22T07:03:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.256+0000 7f7d46a55640 1 -- start start 2024-04-22T07:03:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d46a55640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d40071ce0 0x7f7d40154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d46a55640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d400c8790 0x7f7d401de7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d46a55640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d4014def0 0x7f7d401ef810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d3ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d40071ce0 0x7f7d40154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d3ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d40071ce0 0x7f7d40154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42130/0 (socket says 172.21.15.92:42130) 2024-04-22T07:03:48.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d46a55640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7d400c72b0 con 0x7f7d40071ce0 2024-04-22T07:03:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d3ffff640 1 -- 172.21.15.92:0/41564543 learned_addr learned my addr 172.21.15.92:0/41564543 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:03:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d46a55640 1 -- 172.21.15.92:0/41564543 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7d400c6fd0 con 0x7f7d4014def0 2024-04-22T07:03:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d46a55640 1 -- 172.21.15.92:0/41564543 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7d400c7590 con 0x7f7d400c8790 2024-04-22T07:03:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d3f7fe640 1 --2- 172.21.15.92:0/41564543 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d400c8790 0x7f7d401de7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d44ff2640 1 --2- 172.21.15.92:0/41564543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d4014def0 0x7f7d401ef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d3ffff640 1 -- 172.21.15.92:0/41564543 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d400c8790 msgr2=0x7f7d401de7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d3ffff640 1 --2- 172.21.15.92:0/41564543 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d400c8790 0x7f7d401de7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d3ffff640 1 -- 172.21.15.92:0/41564543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d4014def0 msgr2=0x7f7d401ef810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:48.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d3ffff640 1 --2- 172.21.15.92:0/41564543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d4014def0 0x7f7d401ef810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d3ffff640 1 -- 172.21.15.92:0/41564543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d40155260 con 0x7f7d40071ce0 2024-04-22T07:03:48.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d3f7fe640 1 --2- 172.21.15.92:0/41564543 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d400c8790 0x7f7d401de7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:03:48.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.257+0000 7f7d3ffff640 1 --2- 172.21.15.92:0/41564543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d40071ce0 0x7f7d40154c40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000c820 tx=0x7f7d3000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:48.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.258+0000 7f7d3cff9640 1 -- 172.21.15.92:0/41564543 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30015070 con 0x7f7d40071ce0 2024-04-22T07:03:48.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.258+0000 7f7d3cff9640 1 -- 172.21.15.92:0/41564543 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d30007a30 con 0x7f7d40071ce0 2024-04-22T07:03:48.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.258+0000 7f7d3cff9640 1 -- 172.21.15.92:0/41564543 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30007e00 con 0x7f7d40071ce0 2024-04-22T07:03:48.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.258+0000 7f7d46a55640 1 -- 172.21.15.92:0/41564543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d401519f0 con 0x7f7d40071ce0 2024-04-22T07:03:48.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.258+0000 7f7d46a55640 1 -- 172.21.15.92:0/41564543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d40151fb0 con 0x7f7d40071ce0 2024-04-22T07:03:48.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.258+0000 7f7d3cff9640 1 -- 172.21.15.92:0/41564543 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7d30018070 con 0x7f7d40071ce0 2024-04-22T07:03:48.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.258+0000 7f7d3cff9640 1 -- 172.21.15.92:0/41564543 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7d30011d50 con 0x7f7d40071ce0 2024-04-22T07:03:48.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.259+0000 7f7d46a55640 1 -- 172.21.15.92:0/41564543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d400c8570 con 0x7f7d40071ce0 2024-04-22T07:03:48.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.261+0000 7f7d3cff9640 1 -- 172.21.15.92:0/41564543 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7d30011620 con 0x7f7d40071ce0 2024-04-22T07:03:48.269 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:48.271+0000 7efe53c41040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T07:03:48.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.321+0000 7f7d46a55640 1 -- 172.21.15.92:0/41564543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d401e0320 con 0x7f7d40071ce0 2024-04-22T07:03:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.322+0000 7f7d3cff9640 1 -- 172.21.15.92:0/41564543 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7d30011800 con 0x7f7d40071ce0 2024-04-22T07:03:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:48.320 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":10,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:03:48.098217+0000","last_up_change":"0.000000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:03:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.323+0000 7f7d46a55640 1 -- 172.21.15.92:0/41564543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d40071ce0 msgr2=0x7f7d40154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.323+0000 7f7d46a55640 1 --2- 172.21.15.92:0/41564543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d40071ce0 0x7f7d40154c40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000c820 tx=0x7f7d3000cc20 comp rx=0 tx=0).stop 2024-04-22T07:03:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.323+0000 7f7d46a55640 1 -- 172.21.15.92:0/41564543 shutdown_connections 2024-04-22T07:03:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.323+0000 7f7d46a55640 1 --2- 172.21.15.92:0/41564543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d4014def0 0x7f7d401ef810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.323+0000 7f7d46a55640 1 --2- 172.21.15.92:0/41564543 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d400c8790 0x7f7d401de7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.323+0000 7f7d46a55640 1 --2- 172.21.15.92:0/41564543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d40071ce0 0x7f7d40154c40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.323+0000 7f7d46a55640 1 -- 172.21.15.92:0/41564543 >> 172.21.15.92:0/41564543 conn(0x7f7d4006f8d0 msgr2=0x7f7d40073660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.323+0000 7f7d46a55640 1 -- 172.21.15.92:0/41564543 shutdown_connections 2024-04-22T07:03:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:48.323+0000 7f7d46a55640 1 -- 172.21.15.92:0/41564543 wait complete. 2024-04-22T07:03:48.332 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-22T07:03:48.332 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-22T07:03:48.350 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:48.352+0000 7fdde9638040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T07:03:48.403 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:48.405+0000 7fdde9638040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T07:03:48.523 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:48.525+0000 7fdde9638040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T07:03:48.525 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:48.528+0000 7efe53c41040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T07:03:48.577 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:48.579+0000 7efe53c41040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T07:03:48.691 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:48.693+0000 7efe53c41040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T07:03:48.723 INFO:tasks.ceph.osd.5.smithi145.stderr:2024-04-22T07:03:48.725+0000 7f2de0525640 -1 Falling back to public interface 2024-04-22T07:03:48.723 INFO:tasks.ceph.osd.4.smithi145.stderr:2024-04-22T07:03:48.726+0000 7ff1e5331640 -1 Falling back to public interface 2024-04-22T07:03:48.723 INFO:tasks.ceph.osd.4.smithi145.stderr:2024-04-22T07:03:48.726+0000 7ff1e42e0640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6803/0: (98) Address already in use 2024-04-22T07:03:48.725 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-22T07:03:48.727+0000 7fc97f013640 -1 Falling back to public interface 2024-04-22T07:03:48.735 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-22T07:03:48.737+0000 7fe9e920d640 -1 Falling back to public interface 2024-04-22T07:03:48.739 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-22T07:03:48.741+0000 7f72e2180640 -1 Falling back to public interface 2024-04-22T07:03:48.739 INFO:tasks.ceph.osd.6.smithi145.stderr:2024-04-22T07:03:48.742+0000 7f8712412640 -1 Falling back to public interface 2024-04-22T07:03:48.743 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-22T07:03:48.745+0000 7f238c40d640 -1 Falling back to public interface 2024-04-22T07:03:48.753 INFO:tasks.ceph.osd.7.smithi145.stderr:2024-04-22T07:03:48.756+0000 7fb643933640 -1 Falling back to public interface 2024-04-22T07:03:48.805 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:48.808+0000 7fdde9638040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T07:03:48.853 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:48.855+0000 7fdde9638040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T07:03:48.973 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:48.975+0000 7efe53c41040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T07:03:49.015 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:49.012+0000 7fdde9638040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T07:03:49.021 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:49.024+0000 7efe53c41040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T07:03:49.137 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-22T07:03:49.147 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:49.149+0000 7fdde9638040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T07:03:49.175 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:49.178+0000 7efe53c41040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T07:03:49.200 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:49.201+0000 7fdde9638040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T07:03:49.227 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.228+0000 7f6eee72b640 1 Processor -- start 2024-04-22T07:03:49.227 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.228+0000 7f6eee72b640 1 -- start start 2024-04-22T07:03:49.229 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6eee72b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6ee8151d80 0x7f6ee815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:49.229 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6eee72b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ee8070f40 0x7f6ee80c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:49.229 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6eee72b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee80703e0 con 0x7f6ee80c75e0 2024-04-22T07:03:49.229 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6eee72b640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee80c4930 con 0x7f6ee8070f40 2024-04-22T07:03:49.229 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6eee72b640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee80c4c10 con 0x7f6ee8151d80 2024-04-22T07:03:49.229 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6eeccc8640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ee8070f40 0x7f6ee80c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:49.229 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6eeccc8640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ee8070f40 0x7f6ee80c7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42134/0 (socket says 172.21.15.92:42134) 2024-04-22T07:03:49.229 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6eeccc8640 1 -- 172.21.15.92:0/1555078785 learned_addr learned my addr 172.21.15.92:0/1555078785 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:03:49.229 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6ee77fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6ee80c75e0 0x7f6ee80cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55778/0 (socket says 172.21.15.92:55778) 2024-04-22T07:03:49.229 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6ee7fff640 1 --2- 172.21.15.92:0/1555078785 >> v2:172.21.15.145:3300/0 conn(0x7f6ee8151d80 0x7f6ee815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:49.229 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6eeccc8640 1 -- 172.21.15.92:0/1555078785 >> v2:172.21.15.145:3300/0 conn(0x7f6ee8151d80 msgr2=0x7f6ee815ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:49.230 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6eeccc8640 1 --2- 172.21.15.92:0/1555078785 >> v2:172.21.15.145:3300/0 conn(0x7f6ee8151d80 0x7f6ee815ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.230 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6eeccc8640 1 -- 172.21.15.92:0/1555078785 >> v1:172.21.15.92:6789/0 conn(0x7f6ee80c75e0 legacy=0x7f6ee80cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:49.230 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.229+0000 7f6eeccc8640 1 -- 172.21.15.92:0/1555078785 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee815af80 con 0x7f6ee8070f40 2024-04-22T07:03:49.230 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.230+0000 7f6eeccc8640 1 --2- 172.21.15.92:0/1555078785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8070f40 0x7f6ee80c7d00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009e50 tx=0x7f6ee002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cc04913552dd37 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:49.230 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.230+0000 7f6ee6ffd640 1 -- 172.21.15.92:0/1555078785 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee002ca80 con 0x7f6ee8070f40 2024-04-22T07:03:49.230 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.230+0000 7f6ee6ffd640 1 -- 172.21.15.92:0/1555078785 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee002cc20 con 0x7f6ee8070f40 2024-04-22T07:03:49.230 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.230+0000 7f6ee6ffd640 1 -- 172.21.15.92:0/1555078785 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00343f0 con 0x7f6ee8070f40 2024-04-22T07:03:49.233 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.230+0000 7f6eee72b640 1 -- 172.21.15.92:0/1555078785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8070f40 msgr2=0x7f6ee80c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:49.233 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.230+0000 7f6eee72b640 1 --2- 172.21.15.92:0/1555078785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8070f40 0x7f6ee80c7d00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009e50 tx=0x7f6ee002bee0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.233 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.231+0000 7f6eee72b640 1 -- 172.21.15.92:0/1555078785 shutdown_connections 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.231+0000 7f6eee72b640 1 --2- 172.21.15.92:0/1555078785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8070f40 0x7f6ee80c7d00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.231+0000 7f6eee72b640 1 --2- 172.21.15.92:0/1555078785 >> v2:172.21.15.145:3300/0 conn(0x7f6ee8151d80 0x7f6ee815ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.231+0000 7f6eee72b640 1 -- 172.21.15.92:0/1555078785 >> 172.21.15.92:0/1555078785 conn(0x7f6ee8072810 msgr2=0x7f6ee8072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.231+0000 7f6eee72b640 1 -- 172.21.15.92:0/1555078785 shutdown_connections 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.231+0000 7f6eee72b640 1 -- 172.21.15.92:0/1555078785 wait complete. 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.232+0000 7f6eee72b640 1 Processor -- start 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.232+0000 7f6eee72b640 1 -- start start 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.232+0000 7f6eee72b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ee8070f40 0x7f6ee81e1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6eee72b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee80c75e0 0x7f6ee806b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6eee72b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ee8151d80 0x7f6ee806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6eee72b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee80cd960 con 0x7f6ee80c75e0 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6eee72b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee80c4ef0 con 0x7f6ee8151d80 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6eee72b640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee80cdae0 con 0x7f6ee8070f40 2024-04-22T07:03:49.234 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee80c75e0 0x7f6ee806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee80c75e0 0x7f6ee806b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42138/0 (socket says 172.21.15.92:42138) 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee77fe640 1 -- 172.21.15.92:0/4010157625 learned_addr learned my addr 172.21.15.92:0/4010157625 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6eeccc8640 1 --2- 172.21.15.92:0/4010157625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ee8151d80 0x7f6ee806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee7fff640 1 --2- 172.21.15.92:0/4010157625 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ee8070f40 0x7f6ee81e1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee77fe640 1 -- 172.21.15.92:0/4010157625 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ee8070f40 msgr2=0x7f6ee81e1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee77fe640 1 --2- 172.21.15.92:0/4010157625 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ee8070f40 0x7f6ee81e1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee77fe640 1 -- 172.21.15.92:0/4010157625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ee8151d80 msgr2=0x7f6ee806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee77fe640 1 --2- 172.21.15.92:0/4010157625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ee8151d80 0x7f6ee806beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee77fe640 1 -- 172.21.15.92:0/4010157625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee81f04d0 con 0x7f6ee80c75e0 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee7fff640 1 --2- 172.21.15.92:0/4010157625 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ee8070f40 0x7f6ee81e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee77fe640 1 --2- 172.21.15.92:0/4010157625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee80c75e0 0x7f6ee806b970 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800c470 tx=0x7f6ed8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee4ff9640 1 -- 172.21.15.92:0/4010157625 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80074e0 con 0x7f6ee80c75e0 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee4ff9640 1 -- 172.21.15.92:0/4010157625 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed8007680 con 0x7f6ee80c75e0 2024-04-22T07:03:49.235 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6eee72b640 1 -- 172.21.15.92:0/4010157625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee81e3090 con 0x7f6ee80c75e0 2024-04-22T07:03:49.236 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee4ff9640 1 -- 172.21.15.92:0/4010157625 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8004a20 con 0x7f6ee80c75e0 2024-04-22T07:03:49.236 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6eee72b640 1 -- 172.21.15.92:0/4010157625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee81f0a70 con 0x7f6ee80c75e0 2024-04-22T07:03:49.236 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee4ff9640 1 -- 172.21.15.92:0/4010157625 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6ed801a020 con 0x7f6ee80c75e0 2024-04-22T07:03:49.236 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.233+0000 7f6ee4ff9640 1 -- 172.21.15.92:0/4010157625 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6ed8012ac0 con 0x7f6ee80c75e0 2024-04-22T07:03:49.236 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.234+0000 7f6eee72b640 1 -- 172.21.15.92:0/4010157625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee8069430 con 0x7f6ee80c75e0 2024-04-22T07:03:49.236 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.236+0000 7f6ee4ff9640 1 -- 172.21.15.92:0/4010157625 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6ed8021070 con 0x7f6ee80c75e0 2024-04-22T07:03:49.274 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:49.275+0000 7fdde9638040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T07:03:49.297 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.298+0000 7f6eee72b640 1 -- 172.21.15.92:0/4010157625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ee8070d20 con 0x7f6ee80c75e0 2024-04-22T07:03:49.297 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.298+0000 7f6ee4ff9640 1 -- 172.21.15.92:0/4010157625 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f6ed8012e20 con 0x7f6ee80c75e0 2024-04-22T07:03:49.297 INFO:teuthology.misc.health.smithi092.stdout: 2024-04-22T07:03:49.297 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":10,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:03:48.098217+0000","last_up_change":"0.000000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:03:49.299 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.299+0000 7f6eee72b640 1 -- 172.21.15.92:0/4010157625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee80c75e0 msgr2=0x7f6ee806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:49.299 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.299+0000 7f6eee72b640 1 --2- 172.21.15.92:0/4010157625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee80c75e0 0x7f6ee806b970 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800c470 tx=0x7f6ed8001e70 comp rx=0 tx=0).stop 2024-04-22T07:03:49.299 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.299+0000 7f6eee72b640 1 -- 172.21.15.92:0/4010157625 shutdown_connections 2024-04-22T07:03:49.299 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.299+0000 7f6eee72b640 1 --2- 172.21.15.92:0/4010157625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ee8151d80 0x7f6ee806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.299 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.299+0000 7f6eee72b640 1 --2- 172.21.15.92:0/4010157625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee80c75e0 0x7f6ee806b970 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.299 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.299+0000 7f6eee72b640 1 --2- 172.21.15.92:0/4010157625 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ee8070f40 0x7f6ee81e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.299 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.299+0000 7f6eee72b640 1 -- 172.21.15.92:0/4010157625 >> 172.21.15.92:0/4010157625 conn(0x7f6ee8072810 msgr2=0x7f6ee80c92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:49.299 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.299+0000 7f6eee72b640 1 -- 172.21.15.92:0/4010157625 shutdown_connections 2024-04-22T07:03:49.299 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:49.299+0000 7f6eee72b640 1 -- 172.21.15.92:0/4010157625 wait complete. 2024-04-22T07:03:49.307 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-22T07:03:49.312 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:49.314+0000 7efe53c41040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T07:03:49.322 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:49.323+0000 7fdde9638040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T07:03:49.360 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:49.363+0000 7efe53c41040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T07:03:49.428 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:49.431+0000 7efe53c41040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T07:03:49.472 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:49.475+0000 7efe53c41040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T07:03:49.804 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:49.805+0000 7fdde9638040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T07:03:49.922 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:49.923+0000 7fdde9638040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T07:03:49.951 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:49.954+0000 7efe53c41040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T07:03:50.066 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-04-22T07:03:50.069+0000 7efe53c41040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T07:03:50.935 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:50.936+0000 7fdd8a1d9640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-22T07:03:50.935 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:03:50.936+0000 7fdd8a1d9640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-22T07:03:52.163 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-22T07:03:52.166+0000 7fc97f013640 -1 osd.3 0 log_to_monitors true 2024-04-22T07:03:52.205 INFO:tasks.ceph.osd.4.smithi145.stderr:2024-04-22T07:03:52.207+0000 7ff1e5331640 -1 osd.4 0 log_to_monitors true 2024-04-22T07:03:52.221 INFO:tasks.ceph.osd.5.smithi145.stderr:2024-04-22T07:03:52.224+0000 7f2de0525640 -1 osd.5 0 log_to_monitors true 2024-04-22T07:03:52.222 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-22T07:03:52.224+0000 7fe9e920d640 -1 osd.2 0 log_to_monitors true 2024-04-22T07:03:52.224 INFO:tasks.ceph.osd.6.smithi145.stderr:2024-04-22T07:03:52.227+0000 7f8712412640 -1 osd.6 0 log_to_monitors true 2024-04-22T07:03:52.249 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-22T07:03:52.252+0000 7f72e2180640 -1 osd.1 0 log_to_monitors true 2024-04-22T07:03:52.255 INFO:tasks.ceph.osd.7.smithi145.stderr:2024-04-22T07:03:52.257+0000 7fb643933640 -1 osd.7 0 log_to_monitors true 2024-04-22T07:03:52.259 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-22T07:03:52.262+0000 7f238c40d640 -1 osd.0 0 log_to_monitors true 2024-04-22T07:03:55.745 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-22T07:03:55.747+0000 7f72de116640 -1 osd.1 0 waiting for initial osdmap 2024-04-22T07:03:56.222 INFO:tasks.ceph.osd.4.smithi145.stderr:2024-04-22T07:03:56.017+0000 7ff1e12c7640 -1 osd.4 0 waiting for initial osdmap 2024-04-22T07:03:56.222 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-22T07:03:55.763+0000 7f72d5f36640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:56.223 INFO:tasks.ceph.osd.5.smithi145.stderr:2024-04-22T07:03:56.026+0000 7f2ddcccf640 -1 osd.5 0 waiting for initial osdmap 2024-04-22T07:03:56.223 INFO:tasks.ceph.osd.6.smithi145.stderr:2024-04-22T07:03:56.027+0000 7f870e665640 -1 osd.6 0 waiting for initial osdmap 2024-04-22T07:03:56.223 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-22T07:03:55.999+0000 7fe9e4ae0640 -1 osd.2 0 waiting for initial osdmap 2024-04-22T07:03:56.224 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-22T07:03:56.015+0000 7fe9dc900640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:56.224 INFO:tasks.ceph.osd.7.smithi145.stderr:2024-04-22T07:03:56.029+0000 7fb63f8c9640 -1 osd.7 0 waiting for initial osdmap 2024-04-22T07:03:56.224 INFO:tasks.ceph.osd.4.smithi145.stderr:2024-04-22T07:03:56.032+0000 7ff1d90e7640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:56.224 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-22T07:03:56.046+0000 7f23884f4640 -1 osd.0 0 waiting for initial osdmap 2024-04-22T07:03:56.225 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-22T07:03:56.063+0000 7f237fb00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:56.225 INFO:tasks.ceph.osd.5.smithi145.stderr:2024-04-22T07:03:56.040+0000 7f2dd42db640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:56.226 INFO:tasks.ceph.osd.6.smithi145.stderr:2024-04-22T07:03:56.040+0000 7f8705c71640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:56.226 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-22T07:03:56.069+0000 7fc97b1be640 -1 osd.3 0 waiting for initial osdmap 2024-04-22T07:03:56.226 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-22T07:03:56.081+0000 7fc9727ca640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:56.226 INFO:tasks.ceph.osd.7.smithi145.stderr:2024-04-22T07:03:56.045+0000 7fb6376e9640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:56.523 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-22T07:03:56.613 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.614+0000 7f1652166640 1 Processor -- start 2024-04-22T07:03:56.613 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.614+0000 7f1652166640 1 -- start start 2024-04-22T07:03:56.614 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.615+0000 7f1652166640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f164c1582a0 0x7f164c06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:56.614 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.615+0000 7f1652166640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f164c15bcd0 con 0x7f164c06e800 2024-04-22T07:03:56.614 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.615+0000 7f1652166640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f164c070180 con 0x7f164c070db0 2024-04-22T07:03:56.614 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.615+0000 7f1652166640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f164c15b7b0 con 0x7f164c1582a0 2024-04-22T07:03:56.614 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.615+0000 7f164b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f164c1582a0 0x7f164c06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:56.614 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.615+0000 7f164affd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f164c06e800 0x7f164c0ce480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55786/0 (socket says 172.21.15.92:55786) 2024-04-22T07:03:56.614 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.615+0000 7f164affd640 1 -- 172.21.15.92:0/1403903269 learned_addr learned my addr 172.21.15.92:0/1403903269 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:03:56.614 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.615+0000 7f164b7fe640 1 -- 172.21.15.92:0/1403903269 >> v1:172.21.15.145:6789/0 conn(0x7f164c070db0 legacy=0x7f164c165600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:56.614 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.615+0000 7f164b7fe640 1 -- 172.21.15.92:0/1403903269 >> v1:172.21.15.92:6789/0 conn(0x7f164c06e800 legacy=0x7f164c0ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:56.614 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.615+0000 7f164b7fe640 1 -- 172.21.15.92:0/1403903269 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164c0b0f70 con 0x7f164c1582a0 2024-04-22T07:03:56.615 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.616+0000 7f164b7fe640 1 --2- 172.21.15.92:0/1403903269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f164c1582a0 0x7f164c06cb80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f163c0097a0 tx=0x7f163c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06172fe5a0741c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:56.616 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.616+0000 7f164a7fc640 1 -- 172.21.15.92:0/1403903269 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c034070 con 0x7f164c1582a0 2024-04-22T07:03:56.616 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.616+0000 7f164a7fc640 1 -- 172.21.15.92:0/1403903269 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163c030610 con 0x7f164c1582a0 2024-04-22T07:03:56.616 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.616+0000 7f164a7fc640 1 -- 172.21.15.92:0/1403903269 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c0309e0 con 0x7f164c1582a0 2024-04-22T07:03:56.616 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.617+0000 7f1652166640 1 -- 172.21.15.92:0/1403903269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f164c1582a0 msgr2=0x7f164c06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:56.616 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.617+0000 7f1652166640 1 --2- 172.21.15.92:0/1403903269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f164c1582a0 0x7f164c06cb80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f163c0097a0 tx=0x7f163c02b960 comp rx=0 tx=0).stop 2024-04-22T07:03:56.616 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.617+0000 7f1652166640 1 -- 172.21.15.92:0/1403903269 shutdown_connections 2024-04-22T07:03:56.616 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.617+0000 7f1652166640 1 --2- 172.21.15.92:0/1403903269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f164c1582a0 0x7f164c06cb80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:56.616 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.617+0000 7f1652166640 1 -- 172.21.15.92:0/1403903269 >> 172.21.15.92:0/1403903269 conn(0x7f164c074bb0 msgr2=0x7f164c074fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:56.616 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.617+0000 7f1652166640 1 -- 172.21.15.92:0/1403903269 shutdown_connections 2024-04-22T07:03:56.616 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.617+0000 7f1652166640 1 -- 172.21.15.92:0/1403903269 wait complete. 2024-04-22T07:03:56.616 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.618+0000 7f1652166640 1 Processor -- start 2024-04-22T07:03:56.618 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.618+0000 7f1652166640 1 -- start start 2024-04-22T07:03:56.619 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f1652166640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f164c06e800 0x7f164c154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:56.619 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f1652166640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f164c070db0 0x7f164c0cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:56.619 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f1652166640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f164c1582a0 0x7f164c0cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:56.619 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f1652166640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f164c070880 con 0x7f164c06e800 2024-04-22T07:03:56.619 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f1652166640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f164c0c8f30 con 0x7f164c070db0 2024-04-22T07:03:56.619 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f1652166640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f164c0c9690 con 0x7f164c1582a0 2024-04-22T07:03:56.619 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f164b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f164c06e800 0x7f164c154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:56.619 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f164b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f164c06e800 0x7f164c154980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42256/0 (socket says 172.21.15.92:42256) 2024-04-22T07:03:56.619 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f164b7fe640 1 -- 172.21.15.92:0/2464881915 learned_addr learned my addr 172.21.15.92:0/2464881915 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:03:56.619 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f164affd640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f164c070db0 0x7f164c0cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:56.619 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f164bfff640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f164c1582a0 0x7f164c0cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:56.619 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f164b7fe640 1 -- 172.21.15.92:0/2464881915 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f164c1582a0 msgr2=0x7f164c0cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f164b7fe640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f164c1582a0 0x7f164c0cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f164b7fe640 1 -- 172.21.15.92:0/2464881915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f164c070db0 msgr2=0x7f164c0cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f164b7fe640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f164c070db0 0x7f164c0cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f164b7fe640 1 -- 172.21.15.92:0/2464881915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164c1f1470 con 0x7f164c06e800 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f164bfff640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f164c1582a0 0x7f164c0cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f164b7fe640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f164c06e800 0x7f164c154980 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f163c02bf70 tx=0x7f163c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.619+0000 7f1633fff640 1 -- 172.21.15.92:0/2464881915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c034040 con 0x7f164c06e800 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.620+0000 7f1633fff640 1 -- 172.21.15.92:0/2464881915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163c039a10 con 0x7f164c06e800 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.620+0000 7f1633fff640 1 -- 172.21.15.92:0/2464881915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c039e20 con 0x7f164c06e800 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.620+0000 7f1652166640 1 -- 172.21.15.92:0/2464881915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164c155640 con 0x7f164c06e800 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.620+0000 7f1652166640 1 -- 172.21.15.92:0/2464881915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f164c0b28d0 con 0x7f164c06e800 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.620+0000 7f1652166640 1 -- 172.21.15.92:0/2464881915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f164c069430 con 0x7f164c06e800 2024-04-22T07:03:56.620 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.621+0000 7f1633fff640 1 -- 172.21.15.92:0/2464881915 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f163c037070 con 0x7f164c06e800 2024-04-22T07:03:56.621 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.621+0000 7f1633fff640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f163806c860 0x7f163806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:56.621 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.621+0000 7f1633fff640 1 -- 172.21.15.92:0/2464881915 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f163c0acd90 con 0x7f164c06e800 2024-04-22T07:03:56.621 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.621+0000 7f164affd640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f163806c860 0x7f163806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:56.621 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.622+0000 7f164affd640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f163806c860 0x7f163806ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f164c0c4050 tx=0x7f1640009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:56.624 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.626+0000 7f1633fff640 1 -- 172.21.15.92:0/2464881915 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f163c0b0030 con 0x7f164c06e800 2024-04-22T07:03:56.807 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.809+0000 7f1652166640 1 -- 172.21.15.92:0/2464881915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f164c0b09c0 con 0x7f164c06e800 2024-04-22T07:03:56.808 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.810+0000 7f1633fff640 1 -- 172.21.15.92:0/2464881915 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10861 (secure 0 0 0) 0x7f163c07c030 con 0x7f164c06e800 2024-04-22T07:03:56.808 INFO:teuthology.misc.health.smithi092.stdout: 2024-04-22T07:03:56.808 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":13,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:03:56.750732+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:03:56.810 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.812+0000 7f1652166640 1 -- 172.21.15.92:0/2464881915 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f163806c860 msgr2=0x7f163806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:56.810 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.812+0000 7f1652166640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f163806c860 0x7f163806ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f164c0c4050 tx=0x7f1640009040 comp rx=0 tx=0).stop 2024-04-22T07:03:56.810 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.812+0000 7f1652166640 1 -- 172.21.15.92:0/2464881915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f164c06e800 msgr2=0x7f164c154980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:56.810 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.812+0000 7f1652166640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f164c06e800 0x7f164c154980 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f163c02bf70 tx=0x7f163c0046c0 comp rx=0 tx=0).stop 2024-04-22T07:03:56.810 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.812+0000 7f1652166640 1 -- 172.21.15.92:0/2464881915 shutdown_connections 2024-04-22T07:03:56.810 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.812+0000 7f1652166640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f163806c860 0x7f163806ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:56.810 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.812+0000 7f1652166640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f164c1582a0 0x7f164c0cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:56.810 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.812+0000 7f1652166640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f164c070db0 0x7f164c0cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:56.811 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.812+0000 7f1652166640 1 --2- 172.21.15.92:0/2464881915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f164c06e800 0x7f164c154980 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:56.811 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.812+0000 7f1652166640 1 -- 172.21.15.92:0/2464881915 >> 172.21.15.92:0/2464881915 conn(0x7f164c074bb0 msgr2=0x7f164c06cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:56.811 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.812+0000 7f1652166640 1 -- 172.21.15.92:0/2464881915 shutdown_connections 2024-04-22T07:03:56.811 INFO:teuthology.misc.health.smithi092.stderr:2024-04-22T07:03:56.812+0000 7f1652166640 1 -- 172.21.15.92:0/2464881915 wait complete. 2024-04-22T07:03:56.820 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-22T07:03:56.820 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-22T07:03:56.820 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-22T07:03:56.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-22T07:03:56.823 INFO:tasks.ceph.mds.a:Started 2024-04-22T07:03:56.823 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-22T07:03:56.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-22T07:03:56.825 INFO:tasks.ceph.mds.c:Started 2024-04-22T07:03:56.825 INFO:tasks.ceph.mds.e:Restarting daemon 2024-04-22T07:03:56.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-04-22T07:03:56.827 INFO:tasks.ceph.mds.e:Started 2024-04-22T07:03:56.828 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-22T07:03:56.828 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-22T07:03:56.831 INFO:tasks.ceph.mds.b:Started 2024-04-22T07:03:56.831 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-22T07:03:56.831 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-22T07:03:56.833 INFO:tasks.ceph.mds.d:Started 2024-04-22T07:03:56.833 INFO:tasks.ceph.mds.f:Restarting daemon 2024-04-22T07:03:56.833 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-04-22T07:03:56.835 INFO:tasks.ceph.mds.f:Started 2024-04-22T07:03:56.835 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-22T07:03:56.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:03:56.897 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2024-04-22T07:03:56.897 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2024-04-22T07:03:56.901 INFO:tasks.ceph.mds.e.smithi092.stdout:starting mds.e at 2024-04-22T07:03:56.914 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-04-22T07:03:56.914 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-04-22T07:03:56.915 INFO:tasks.ceph.mds.f.smithi145.stdout:starting mds.f at 2024-04-22T07:03:56.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.941+0000 7fc1ce6a5640 1 Processor -- start 2024-04-22T07:03:56.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.941+0000 7fc1ce6a5640 1 -- start start 2024-04-22T07:03:56.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.942+0000 7fc1ce6a5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1c80d0f50 0x7fc1c815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:56.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.942+0000 7fc1ce6a5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c815ab30 con 0x7fc1c80d0b70 2024-04-22T07:03:56.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.942+0000 7fc1ce6a5640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c815adf0 con 0x7fc1c80cb120 2024-04-22T07:03:56.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.942+0000 7fc1ce6a5640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c815ba00 con 0x7fc1c80d0f50 2024-04-22T07:03:56.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.942+0000 7fc1c77fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1c80d0f50 0x7fc1c815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:56.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.942+0000 7fc1c7fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc1c80d0b70 0x7fc1c8154a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55792/0 (socket says 172.21.15.92:55792) 2024-04-22T07:03:56.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.942+0000 7fc1c7fff640 1 -- 172.21.15.92:0/2311699875 learned_addr learned my addr 172.21.15.92:0/2311699875 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:03:56.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.943+0000 7fc1c77fe640 1 -- 172.21.15.92:0/2311699875 >> v1:172.21.15.145:6789/0 conn(0x7fc1c80cb120 legacy=0x7fc1c8165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:56.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.943+0000 7fc1c77fe640 1 -- 172.21.15.92:0/2311699875 >> v1:172.21.15.92:6789/0 conn(0x7fc1c80d0b70 legacy=0x7fc1c8154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:56.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.943+0000 7fc1c77fe640 1 -- 172.21.15.92:0/2311699875 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c815b0d0 con 0x7fc1c80d0f50 2024-04-22T07:03:56.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.943+0000 7fc1c6ffd640 1 -- 172.21.15.92:0/2311699875 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3894042499 0 0) 0x7fc1c815ab30 con 0x7fc1c80d0b70 2024-04-22T07:03:56.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.943+0000 7fc1c77fe640 1 --2- 172.21.15.92:0/2311699875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c80d0f50 0x7fc1c815e2a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0097a0 tx=0x7fc1bc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b16472d3af5541 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:56.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.944+0000 7fc1c6ffd640 1 -- 172.21.15.92:0/2311699875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0315a0 con 0x7fc1c80d0f50 2024-04-22T07:03:56.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.944+0000 7fc1c6ffd640 1 -- 172.21.15.92:0/2311699875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1bc037070 con 0x7fc1c80d0f50 2024-04-22T07:03:56.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.944+0000 7fc1c6ffd640 1 -- 172.21.15.92:0/2311699875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc031960 con 0x7fc1c80d0f50 2024-04-22T07:03:56.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.944+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2311699875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c80d0f50 msgr2=0x7fc1c815e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:56.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.944+0000 7fc1ce6a5640 1 --2- 172.21.15.92:0/2311699875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c80d0f50 0x7fc1c815e2a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0097a0 tx=0x7fc1bc02b840 comp rx=0 tx=0).stop 2024-04-22T07:03:56.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.944+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2311699875 shutdown_connections 2024-04-22T07:03:56.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.944+0000 7fc1ce6a5640 1 --2- 172.21.15.92:0/2311699875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c80d0f50 0x7fc1c815e2a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:56.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.944+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2311699875 >> 172.21.15.92:0/2311699875 conn(0x7fc1c80736b0 msgr2=0x7fc1c8073ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:56.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.944+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2311699875 shutdown_connections 2024-04-22T07:03:56.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.944+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2311699875 wait complete. 2024-04-22T07:03:56.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.945+0000 7fc1ce6a5640 1 Processor -- start 2024-04-22T07:03:56.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.945+0000 7fc1ce6a5640 1 -- start start 2024-04-22T07:03:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.946+0000 7fc1ce6a5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c80cb120 0x7fc1c80ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.946+0000 7fc1ce6a5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1c80d0b70 0x7fc1c8143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.946+0000 7fc1ce6a5640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1c80d0f50 0x7fc1c8153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.946+0000 7fc1ce6a5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c80cae00 con 0x7fc1c80cb120 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.946+0000 7fc1ce6a5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c815b690 con 0x7fc1c80d0b70 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.946+0000 7fc1ce6a5640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c80caf80 con 0x7fc1c80d0f50 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.946+0000 7fc1c7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c80cb120 0x7fc1c80ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.946+0000 7fc1c7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c80cb120 0x7fc1c80ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42286/0 (socket says 172.21.15.92:42286) 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.946+0000 7fc1c7fff640 1 -- 172.21.15.92:0/2768360970 learned_addr learned my addr 172.21.15.92:0/2768360970 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.946+0000 7fc1ccc42640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1c80d0f50 0x7fc1c8153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.946+0000 7fc1c77fe640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1c80d0b70 0x7fc1c8143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.947+0000 7fc1c7fff640 1 -- 172.21.15.92:0/2768360970 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1c80d0f50 msgr2=0x7fc1c8153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.947+0000 7fc1c7fff640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1c80d0f50 0x7fc1c8153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.947+0000 7fc1c7fff640 1 -- 172.21.15.92:0/2768360970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1c80d0b70 msgr2=0x7fc1c8143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.947+0000 7fc1c7fff640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1c80d0b70 0x7fc1c8143fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.947+0000 7fc1c7fff640 1 -- 172.21.15.92:0/2768360970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c80cd430 con 0x7fc1c80cb120 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.947+0000 7fc1c7fff640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c80cb120 0x7fc1c80ccef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800c500 tx=0x7fc1b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.947+0000 7fc1c4ff9640 1 -- 172.21.15.92:0/2768360970 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8018070 con 0x7fc1c80cb120 2024-04-22T07:03:56.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.947+0000 7fc1c4ff9640 1 -- 172.21.15.92:0/2768360970 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b8007580 con 0x7fc1c80cb120 2024-04-22T07:03:56.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.947+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2768360970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c8148c70 con 0x7fc1c80cb120 2024-04-22T07:03:56.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.947+0000 7fc1c4ff9640 1 -- 172.21.15.92:0/2768360970 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80049e0 con 0x7fc1c80cb120 2024-04-22T07:03:56.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.947+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2768360970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c8149680 con 0x7fc1c80cb120 2024-04-22T07:03:56.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.948+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2768360970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c8069930 con 0x7fc1c80cb120 2024-04-22T07:03:56.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.949+0000 7fc1c4ff9640 1 -- 172.21.15.92:0/2768360970 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fc1b8012400 con 0x7fc1c80cb120 2024-04-22T07:03:56.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.950+0000 7fc1c4ff9640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc1b406c890 0x7fc1b406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:56.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.950+0000 7fc1c77fe640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc1b406c890 0x7fc1b406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:56.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.950+0000 7fc1c4ff9640 1 -- 172.21.15.92:0/2768360970 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7fc1b808a850 con 0x7fc1c80cb120 2024-04-22T07:03:56.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.950+0000 7fc1c77fe640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc1b406c890 0x7fc1b406ed50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc036000 tx=0x7fc1bc03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:56.955+0000 7fc1c4ff9640 1 -- 172.21.15.92:0/2768360970 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc1b804eb80 con 0x7fc1c80cb120 2024-04-22T07:03:57.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.132+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2768360970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1c815aae0 con 0x7fc1c80cb120 2024-04-22T07:03:57.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.133+0000 7fc1c4ff9640 1 -- 172.21.15.92:0/2768360970 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10861 (secure 0 0 0) 0x7fc1b8058bf0 con 0x7fc1c80cb120 2024-04-22T07:03:57.131 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:57.131 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":13,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:03:56.750732+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:03:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.135+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2768360970 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc1b406c890 msgr2=0x7fc1b406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.135+0000 7fc1ce6a5640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc1b406c890 0x7fc1b406ed50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc036000 tx=0x7fc1bc03d040 comp rx=0 tx=0).stop 2024-04-22T07:03:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.135+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2768360970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c80cb120 msgr2=0x7fc1c80ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.135+0000 7fc1ce6a5640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c80cb120 0x7fc1c80ccef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800c500 tx=0x7fc1b800c9c0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.135+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2768360970 shutdown_connections 2024-04-22T07:03:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.135+0000 7fc1ce6a5640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc1b406c890 0x7fc1b406ed50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.135+0000 7fc1ce6a5640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1c80d0f50 0x7fc1c8153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.135+0000 7fc1ce6a5640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1c80d0b70 0x7fc1c8143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.135+0000 7fc1ce6a5640 1 --2- 172.21.15.92:0/2768360970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c80cb120 0x7fc1c80ccef0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.135+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2768360970 >> 172.21.15.92:0/2768360970 conn(0x7fc1c80736b0 msgr2=0x7fc1c81547d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.135+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2768360970 shutdown_connections 2024-04-22T07:03:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.135+0000 7fc1ce6a5640 1 -- 172.21.15.92:0/2768360970 wait complete. 2024-04-22T07:03:57.143 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-22T07:03:57.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:03:57.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.243+0000 7efc7f798640 1 Processor -- start 2024-04-22T07:03:57.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.243+0000 7efc7f798640 1 -- start start 2024-04-22T07:03:57.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.244+0000 7efc7f798640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc78155470 0x7efc7815b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:57.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.244+0000 7efc7f798640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efc780cd5e0 0x7efc780b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:57.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.244+0000 7efc7f798640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc7815e660 con 0x7efc780cd200 2024-04-22T07:03:57.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.244+0000 7efc7f798640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7efc7815f480 con 0x7efc78155470 2024-04-22T07:03:57.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.244+0000 7efc7f798640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7efc7815e900 con 0x7efc780cd5e0 2024-04-22T07:03:57.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.244+0000 7efc7cd33640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc78155470 0x7efc7815b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:57.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.244+0000 7efc7cd33640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc78155470 0x7efc7815b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42292/0 (socket says 172.21.15.92:42292) 2024-04-22T07:03:57.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.244+0000 7efc7cd33640 1 -- 172.21.15.92:0/1332860271 learned_addr learned my addr 172.21.15.92:0/1332860271 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:03:57.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.245+0000 7efc7dd35640 1 --2- 172.21.15.92:0/1332860271 >> v2:172.21.15.145:3300/0 conn(0x7efc780cd5e0 0x7efc780b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:57.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.245+0000 7efc7cd33640 1 -- 172.21.15.92:0/1332860271 >> v2:172.21.15.145:3300/0 conn(0x7efc780cd5e0 msgr2=0x7efc780b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:57.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.245+0000 7efc7cd33640 1 --2- 172.21.15.92:0/1332860271 >> v2:172.21.15.145:3300/0 conn(0x7efc780cd5e0 0x7efc780b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.245+0000 7efc7cd33640 1 -- 172.21.15.92:0/1332860271 >> v1:172.21.15.145:6789/0 conn(0x7efc780cd200 legacy=0x7efc78154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:57.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.245+0000 7efc7cd33640 1 -- 172.21.15.92:0/1332860271 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7815ebe0 con 0x7efc78155470 2024-04-22T07:03:57.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.245+0000 7efc7cd33640 1 --2- 172.21.15.92:0/1332860271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78155470 0x7efc7815b9b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc7815e830 tx=0x7efc7402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a869ad56c244eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:57.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.246+0000 7efc6ffff640 1 -- 172.21.15.92:0/1332860271 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7402f840 con 0x7efc78155470 2024-04-22T07:03:57.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.246+0000 7efc6ffff640 1 -- 172.21.15.92:0/1332860271 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7402f9e0 con 0x7efc78155470 2024-04-22T07:03:57.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.246+0000 7efc6ffff640 1 -- 172.21.15.92:0/1332860271 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7402fdb0 con 0x7efc78155470 2024-04-22T07:03:57.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.246+0000 7efc7f798640 1 -- 172.21.15.92:0/1332860271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78155470 msgr2=0x7efc7815b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:57.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.246+0000 7efc7f798640 1 --2- 172.21.15.92:0/1332860271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78155470 0x7efc7815b9b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc7815e830 tx=0x7efc7402ba20 comp rx=0 tx=0).stop 2024-04-22T07:03:57.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.246+0000 7efc7f798640 1 -- 172.21.15.92:0/1332860271 shutdown_connections 2024-04-22T07:03:57.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.246+0000 7efc7f798640 1 --2- 172.21.15.92:0/1332860271 >> v2:172.21.15.145:3300/0 conn(0x7efc780cd5e0 0x7efc780b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.246+0000 7efc7f798640 1 --2- 172.21.15.92:0/1332860271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78155470 0x7efc7815b9b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.246+0000 7efc7f798640 1 -- 172.21.15.92:0/1332860271 >> 172.21.15.92:0/1332860271 conn(0x7efc78073bc0 msgr2=0x7efc780728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:57.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.246+0000 7efc7f798640 1 -- 172.21.15.92:0/1332860271 shutdown_connections 2024-04-22T07:03:57.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.246+0000 7efc7f798640 1 -- 172.21.15.92:0/1332860271 wait complete. 2024-04-22T07:03:57.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.247+0000 7efc7f798640 1 Processor -- start 2024-04-22T07:03:57.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.247+0000 7efc7f798640 1 -- start start 2024-04-22T07:03:57.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7f798640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc780cd200 0x7efc78150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:57.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7f798640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efc780cd5e0 0x7efc781f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:57.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7f798640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78155470 0x7efc7814fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:57.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7f798640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efc780cc430 con 0x7efc78155470 2024-04-22T07:03:57.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7f798640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efc7815f1a0 con 0x7efc780cd200 2024-04-22T07:03:57.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7f798640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efc780cc5b0 con 0x7efc780cd5e0 2024-04-22T07:03:57.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7dd35640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78155470 0x7efc7814fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:57.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7dd35640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78155470 0x7efc7814fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42300/0 (socket says 172.21.15.92:42300) 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7dd35640 1 -- 172.21.15.92:0/1665582152 learned_addr learned my addr 172.21.15.92:0/1665582152 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7d534640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc780cd200 0x7efc78150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7cd33640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efc780cd5e0 0x7efc781f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7dd35640 1 -- 172.21.15.92:0/1665582152 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efc780cd5e0 msgr2=0x7efc781f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7dd35640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efc780cd5e0 0x7efc781f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7dd35640 1 -- 172.21.15.92:0/1665582152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc780cd200 msgr2=0x7efc78150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7dd35640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc780cd200 0x7efc78150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.248+0000 7efc7dd35640 1 -- 172.21.15.92:0/1665582152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc78150f50 con 0x7efc78155470 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.249+0000 7efc7dd35640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78155470 0x7efc7814fc10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc70007e60 tx=0x7efc7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.249+0000 7efc6dffb640 1 -- 172.21.15.92:0/1665582152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc70014620 con 0x7efc78155470 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.249+0000 7efc6dffb640 1 -- 172.21.15.92:0/1665582152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc70019070 con 0x7efc78155470 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.249+0000 7efc7f798640 1 -- 172.21.15.92:0/1665582152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc78151230 con 0x7efc78155470 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.249+0000 7efc6dffb640 1 -- 172.21.15.92:0/1665582152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc70014a00 con 0x7efc78155470 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.249+0000 7efc7f798640 1 -- 172.21.15.92:0/1665582152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7814f6f0 con 0x7efc78155470 2024-04-22T07:03:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.250+0000 7efc7f798640 1 -- 172.21.15.92:0/1665582152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc7814f8f0 con 0x7efc78155470 2024-04-22T07:03:57.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.250+0000 7efc6dffb640 1 -- 172.21.15.92:0/1665582152 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7efc70014ba0 con 0x7efc78155470 2024-04-22T07:03:57.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.251+0000 7efc6dffb640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efc5c06c860 0x7efc5c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:57.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.251+0000 7efc6dffb640 1 -- 172.21.15.92:0/1665582152 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7efc7008bd60 con 0x7efc78155470 2024-04-22T07:03:57.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.251+0000 7efc7d534640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efc5c06c860 0x7efc5c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:57.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.251+0000 7efc7d534640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efc5c06c860 0x7efc5c06ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efc68001780 tx=0x7efc68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:57.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.255+0000 7efc6dffb640 1 -- 172.21.15.92:0/1665582152 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc7005a0f0 con 0x7efc78155470 2024-04-22T07:03:57.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.466+0000 7efc7f798640 1 -- 172.21.15.92:0/1665582152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc78069930 con 0x7efc78155470 2024-04-22T07:03:57.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.467+0000 7efc6dffb640 1 -- 172.21.15.92:0/1665582152 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7efc7005e3f0 con 0x7efc78155470 2024-04-22T07:03:57.465 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:57.465 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-22T07:03:57.465 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 2 2024-04-22T07:03:57.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.469+0000 7efc7f798640 1 -- 172.21.15.92:0/1665582152 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efc5c06c860 msgr2=0x7efc5c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:57.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.469+0000 7efc7f798640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efc5c06c860 0x7efc5c06ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efc68001780 tx=0x7efc68009040 comp rx=0 tx=0).stop 2024-04-22T07:03:57.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.469+0000 7efc7f798640 1 -- 172.21.15.92:0/1665582152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78155470 msgr2=0x7efc7814fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:57.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.469+0000 7efc7f798640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78155470 0x7efc7814fc10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc70007e60 tx=0x7efc7000e4b0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.469+0000 7efc7f798640 1 -- 172.21.15.92:0/1665582152 shutdown_connections 2024-04-22T07:03:57.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.469+0000 7efc7f798640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efc5c06c860 0x7efc5c06ed20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.469+0000 7efc7f798640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78155470 0x7efc7814fc10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.469+0000 7efc7f798640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efc780cd5e0 0x7efc781f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.469+0000 7efc7f798640 1 --2- 172.21.15.92:0/1665582152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc780cd200 0x7efc78150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:57.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.469+0000 7efc7f798640 1 -- 172.21.15.92:0/1665582152 >> 172.21.15.92:0/1665582152 conn(0x7efc78073bc0 msgr2=0x7efc78152710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:57.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.469+0000 7efc7f798640 1 -- 172.21.15.92:0/1665582152 shutdown_connections 2024-04-22T07:03:57.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:57.469+0000 7efc7f798640 1 -- 172.21.15.92:0/1665582152 wait complete. 2024-04-22T07:03:59.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:03:59.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.584+0000 7f7ba5832640 1 Processor -- start 2024-04-22T07:03:59.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.584+0000 7f7ba5832640 1 -- start start 2024-04-22T07:03:59.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.585+0000 7f7ba5832640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7ba00c9130 0x7f7ba015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.585+0000 7f7ba5832640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba0071350 con 0x7f7ba0155070 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.585+0000 7f7ba5832640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba0071070 con 0x7f7ba00c7db0 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.585+0000 7f7ba5832640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f7ba0155450 con 0x7f7ba00c9130 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.585+0000 7f7b9effd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7ba0155070 0x7f7ba00676b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39006/0 (socket says 172.21.15.92:39006) 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.585+0000 7f7b9effd640 1 -- 172.21.15.92:0/2152262553 learned_addr learned my addr 172.21.15.92:0/2152262553 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.585+0000 7f7b9f7fe640 1 --2- 172.21.15.92:0/2152262553 >> v2:172.21.15.145:3300/0 conn(0x7f7ba00c9130 0x7f7ba015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.585+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1464211827 0 0) 0x7f7ba0071350 con 0x7f7ba0155070 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.585+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b88003520 con 0x7f7ba0155070 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.585+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 937510356 0 0) 0x7f7b88003520 con 0x7f7ba0155070 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.585+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 >> v2:172.21.15.145:3300/0 conn(0x7f7ba00c9130 msgr2=0x7f7ba015ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.585+0000 7f7b9dffb640 1 --2- 172.21.15.92:0/2152262553 >> v2:172.21.15.145:3300/0 conn(0x7f7ba00c9130 0x7f7ba015ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 >> v1:172.21.15.145:6789/0 conn(0x7f7ba00c7db0 legacy=0x7f7ba01635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba015e330 con 0x7f7ba0155070 2024-04-22T07:03:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f7b980019f0 con 0x7f7ba0155070 2024-04-22T07:03:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 >> v1:172.21.15.92:6789/0 conn(0x7f7ba0155070 legacy=0x7f7ba00676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9dffb640 1 --2- 172.21.15.92:0/2152262553 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b880039b0 0x7f7b88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9dffb640 1 --2- 172.21.15.92:0/2152262553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 0x7f7b88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9dffb640 1 --2- 172.21.15.92:0/2152262553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b880051c0 0x7f7b8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba0155450 con 0x7f7b88004800 2024-04-22T07:03:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8800c070 con 0x7f7b880051c0 2024-04-22T07:03:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8800c270 con 0x7f7b880039b0 2024-04-22T07:03:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9e7fc640 1 --2- 172.21.15.92:0/2152262553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 0x7f7b88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9effd640 1 --2- 172.21.15.92:0/2152262553 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b880039b0 0x7f7b88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9f7fe640 1 --2- 172.21.15.92:0/2152262553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b880051c0 0x7f7b8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9e7fc640 1 -- 172.21.15.92:0/2152262553 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b880039b0 msgr2=0x7f7b88003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9e7fc640 1 --2- 172.21.15.92:0/2152262553 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b880039b0 0x7f7b88003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9e7fc640 1 -- 172.21.15.92:0/2152262553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b880051c0 msgr2=0x7f7b8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9e7fc640 1 --2- 172.21.15.92:0/2152262553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b880051c0 0x7f7b8800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9f7fe640 1 -- 172.21.15.92:0/2152262553 reap_dead start 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.586+0000 7f7b9e7fc640 1 -- 172.21.15.92:0/2152262553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b88003520 con 0x7f7b88004800 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.587+0000 7f7b9e7fc640 1 --2- 172.21.15.92:0/2152262553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 0x7f7b88004c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7b94001720 tx=0x7f7b88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145f4418a3628f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.587+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94001370 con 0x7f7b88004800 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.587+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b940052d0 con 0x7f7b88004800 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.587+0000 7f7b9dffb640 1 -- 172.21.15.92:0/2152262553 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94003830 con 0x7f7b88004800 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.588+0000 7f7ba5832640 1 -- 172.21.15.92:0/2152262553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 msgr2=0x7f7b88004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.588+0000 7f7ba5832640 1 --2- 172.21.15.92:0/2152262553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 0x7f7b88004c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7b94001720 tx=0x7f7b88005830 comp rx=0 tx=0).stop 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.588+0000 7f7ba5832640 1 -- 172.21.15.92:0/2152262553 shutdown_connections 2024-04-22T07:03:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.588+0000 7f7ba5832640 1 --2- 172.21.15.92:0/2152262553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 0x7f7b88004c80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.588+0000 7f7ba5832640 1 -- 172.21.15.92:0/2152262553 >> 172.21.15.92:0/2152262553 conn(0x7f7ba00c9eb0 msgr2=0x7f7ba00ca2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.588+0000 7f7ba5832640 1 -- 172.21.15.92:0/2152262553 shutdown_connections 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.588+0000 7f7ba5832640 1 -- 172.21.15.92:0/2152262553 wait complete. 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.589+0000 7f7ba5832640 1 Processor -- start 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.589+0000 7f7ba5832640 1 -- start start 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.590+0000 7f7ba5832640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 0x7f7ba015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.590+0000 7f7ba5832640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ba0158900 0x7f7ba00c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.590+0000 7f7ba5832640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ba01524e0 0x7f7ba0152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.590+0000 7f7ba5832640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba00c77f0 con 0x7f7b88004800 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.590+0000 7f7ba5832640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba015e8f0 con 0x7f7ba0158900 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.590+0000 7f7ba5832640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba006cef0 con 0x7f7ba01524e0 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.590+0000 7f7b9effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 0x7f7ba015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.590+0000 7f7b9effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 0x7f7ba015b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44928/0 (socket says 172.21.15.92:44928) 2024-04-22T07:03:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.590+0000 7f7b9effd640 1 -- 172.21.15.92:0/2338032278 learned_addr learned my addr 172.21.15.92:0/2338032278 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.590+0000 7f7b9f7fe640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ba01524e0 0x7f7ba0152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.590+0000 7f7b9e7fc640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ba0158900 0x7f7ba00c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.591+0000 7f7b9effd640 1 -- 172.21.15.92:0/2338032278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ba01524e0 msgr2=0x7f7ba0152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.591+0000 7f7b9effd640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ba01524e0 0x7f7ba0152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.591+0000 7f7b9effd640 1 -- 172.21.15.92:0/2338032278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ba0158900 msgr2=0x7f7ba00c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.591+0000 7f7b9effd640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ba0158900 0x7f7ba00c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.591+0000 7f7b9effd640 1 -- 172.21.15.92:0/2338032278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba0153320 con 0x7f7b88004800 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.591+0000 7f7b9f7fe640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ba01524e0 0x7f7ba0152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.591+0000 7f7b9effd640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 0x7f7ba015b1a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7b980079a0 tx=0x7f7b9800a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.591+0000 7f7b87fff640 1 -- 172.21.15.92:0/2338032278 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b980079d0 con 0x7f7b88004800 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.591+0000 7f7b87fff640 1 -- 172.21.15.92:0/2338032278 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b98018070 con 0x7f7b88004800 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.591+0000 7f7ba5832640 1 -- 172.21.15.92:0/2338032278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba015b900 con 0x7f7b88004800 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.591+0000 7f7b87fff640 1 -- 172.21.15.92:0/2338032278 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98013630 con 0x7f7b88004800 2024-04-22T07:03:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.591+0000 7f7ba5832640 1 -- 172.21.15.92:0/2338032278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba0151720 con 0x7f7b88004800 2024-04-22T07:03:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.592+0000 7f7ba5832640 1 -- 172.21.15.92:0/2338032278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba0068b00 con 0x7f7b88004800 2024-04-22T07:03:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.592+0000 7f7b87fff640 1 -- 172.21.15.92:0/2338032278 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f7b980139e0 con 0x7f7b88004800 2024-04-22T07:03:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.593+0000 7f7b87fff640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7b8c06c890 0x7f7b8c06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.593+0000 7f7b87fff640 1 -- 172.21.15.92:0/2338032278 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4857+0+0 (secure 0 0 0) 0x7f7b9808c1e0 con 0x7f7b88004800 2024-04-22T07:03:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.593+0000 7f7b9e7fc640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7b8c06c890 0x7f7b8c06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.593+0000 7f7b9e7fc640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7b8c06c890 0x7f7b8c06ed50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7b94033000 tx=0x7f7b94009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.597+0000 7f7b87fff640 1 -- 172.21.15.92:0/2338032278 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b9805a930 con 0x7f7b88004800 2024-04-22T07:03:59.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.822+0000 7f7ba5832640 1 -- 172.21.15.92:0/2338032278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ba0071350 con 0x7f7b88004800 2024-04-22T07:03:59.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.822+0000 7f7b87fff640 1 -- 172.21.15.92:0/2338032278 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5518 (secure 0 0 0) 0x7f7b9805a230 con 0x7f7b88004800 2024-04-22T07:03:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:03:59.821 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1059009649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1059009649},{"type":"v1","addr":"172.21.15.145:6835","nonce":1059009649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6836/1058270486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1058270486},{"type":"v1","addr":"172.21.15.145:6836","nonce":1058270486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/3258302920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3258302920},{"type":"v1","addr":"172.21.15.145:6837","nonce":3258302920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/101798872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":101798872},{"type":"v1","addr":"172.21.15.92:6837","nonce":101798872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/3493022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3493022212},{"type":"v1","addr":"172.21.15.92:6839","nonce":3493022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-22T07:03:59.821 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 3 2024-04-22T07:03:59.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.824+0000 7f7ba5832640 1 -- 172.21.15.92:0/2338032278 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7b8c06c890 msgr2=0x7f7b8c06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.824+0000 7f7ba5832640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7b8c06c890 0x7f7b8c06ed50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7b94033000 tx=0x7f7b94009030 comp rx=0 tx=0).stop 2024-04-22T07:03:59.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.824+0000 7f7ba5832640 1 -- 172.21.15.92:0/2338032278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 msgr2=0x7f7ba015b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.824+0000 7f7ba5832640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 0x7f7ba015b1a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7b980079a0 tx=0x7f7b9800a730 comp rx=0 tx=0).stop 2024-04-22T07:03:59.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.825+0000 7f7ba5832640 1 -- 172.21.15.92:0/2338032278 shutdown_connections 2024-04-22T07:03:59.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.825+0000 7f7ba5832640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7b8c06c890 0x7f7b8c06ed50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.825+0000 7f7ba5832640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ba01524e0 0x7f7ba0152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.825+0000 7f7ba5832640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ba0158900 0x7f7ba00c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.825+0000 7f7ba5832640 1 --2- 172.21.15.92:0/2338032278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b88004800 0x7f7ba015b1a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.825+0000 7f7ba5832640 1 -- 172.21.15.92:0/2338032278 >> 172.21.15.92:0/2338032278 conn(0x7f7ba00c9eb0 msgr2=0x7f7ba00ca290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:59.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.825+0000 7f7ba5832640 1 -- 172.21.15.92:0/2338032278 shutdown_connections 2024-04-22T07:03:59.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.825+0000 7f7ba5832640 1 -- 172.21.15.92:0/2338032278 wait complete. 2024-04-22T07:03:59.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:03:59.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.939+0000 7f673d843640 1 Processor -- start 2024-04-22T07:03:59.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.939+0000 7f673d843640 1 -- start start 2024-04-22T07:03:59.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f673d843640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f67380d0f50 0x7f673815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f673d843640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67380cb120 0x7f673815e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f673d843640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673815ab30 con 0x7f67380d0b70 2024-04-22T07:03:59.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f673d843640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f673815ba00 con 0x7f67380d0f50 2024-04-22T07:03:59.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f673d843640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f673815adf0 con 0x7f67380cb120 2024-04-22T07:03:59.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f672ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f67380d0f50 0x7f673815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f672ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f67380d0f50 0x7f673815e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44944/0 (socket says 172.21.15.92:44944) 2024-04-22T07:03:59.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f672ffff640 1 -- 172.21.15.92:0/2043157311 learned_addr learned my addr 172.21.15.92:0/2043157311 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:03:59.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f67377fe640 1 --2- 172.21.15.92:0/2043157311 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67380cb120 0x7f673815e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f672ffff640 1 -- 172.21.15.92:0/2043157311 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67380cb120 msgr2=0x7f673815e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:59.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f672ffff640 1 --2- 172.21.15.92:0/2043157311 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67380cb120 0x7f673815e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f672ffff640 1 -- 172.21.15.92:0/2043157311 >> v1:172.21.15.145:6789/0 conn(0x7f67380d0b70 legacy=0x7f6738154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:59.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.940+0000 7f672ffff640 1 -- 172.21.15.92:0/2043157311 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673815b0d0 con 0x7f67380d0f50 2024-04-22T07:03:59.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.941+0000 7f672ffff640 1 --2- 172.21.15.92:0/2043157311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67380d0f50 0x7f673815e2a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6720009870 tx=0x7f672002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66da2abe5ef146aa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:59.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.941+0000 7f67367fc640 1 -- 172.21.15.92:0/2043157311 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67200317c0 con 0x7f67380d0f50 2024-04-22T07:03:59.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.941+0000 7f67367fc640 1 -- 172.21.15.92:0/2043157311 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6720037070 con 0x7f67380d0f50 2024-04-22T07:03:59.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.941+0000 7f67367fc640 1 -- 172.21.15.92:0/2043157311 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720031b80 con 0x7f67380d0f50 2024-04-22T07:03:59.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.942+0000 7f673d843640 1 -- 172.21.15.92:0/2043157311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67380d0f50 msgr2=0x7f673815e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.942+0000 7f673d843640 1 --2- 172.21.15.92:0/2043157311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67380d0f50 0x7f673815e2a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6720009870 tx=0x7f672002b860 comp rx=0 tx=0).stop 2024-04-22T07:03:59.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.942+0000 7f673d843640 1 -- 172.21.15.92:0/2043157311 shutdown_connections 2024-04-22T07:03:59.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.942+0000 7f673d843640 1 --2- 172.21.15.92:0/2043157311 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67380cb120 0x7f673815e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.942+0000 7f673d843640 1 --2- 172.21.15.92:0/2043157311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67380d0f50 0x7f673815e2a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.942+0000 7f673d843640 1 -- 172.21.15.92:0/2043157311 >> 172.21.15.92:0/2043157311 conn(0x7f67380736b0 msgr2=0x7f6738073ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:59.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.942+0000 7f673d843640 1 -- 172.21.15.92:0/2043157311 shutdown_connections 2024-04-22T07:03:59.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.942+0000 7f673d843640 1 -- 172.21.15.92:0/2043157311 wait complete. 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.943+0000 7f673d843640 1 Processor -- start 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.943+0000 7f673d843640 1 -- start start 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.943+0000 7f673d843640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67380cb120 0x7f67380c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.943+0000 7f673d843640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67380d0b70 0x7f67380ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.943+0000 7f673d843640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67380d0f50 0x7f67380cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.943+0000 7f673d843640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f67380cae00 con 0x7f67380d0f50 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.943+0000 7f673d843640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f673815b690 con 0x7f67380cb120 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.943+0000 7f673d843640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f67380caf80 con 0x7f67380d0b70 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f67377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67380d0f50 0x7f67380cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f67377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67380d0f50 0x7f67380cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44950/0 (socket says 172.21.15.92:44950) 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f67377fe640 1 -- 172.21.15.92:0/13908693 learned_addr learned my addr 172.21.15.92:0/13908693 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f672ffff640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67380d0b70 0x7f67380ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f6736ffd640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67380cb120 0x7f67380c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f67377fe640 1 -- 172.21.15.92:0/13908693 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67380d0b70 msgr2=0x7f67380ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f67377fe640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67380d0b70 0x7f67380ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f67377fe640 1 -- 172.21.15.92:0/13908693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67380cb120 msgr2=0x7f67380c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f67377fe640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67380cb120 0x7f67380c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f67377fe640 1 -- 172.21.15.92:0/13908693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67381f1140 con 0x7f67380d0f50 2024-04-22T07:03:59.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f67377fe640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67380d0f50 0x7f67380cf0b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f673000efc0 tx=0x7f673000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:59.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f6734ff9640 1 -- 172.21.15.92:0/13908693 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730018020 con 0x7f67380d0f50 2024-04-22T07:03:59.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f6734ff9640 1 -- 172.21.15.92:0/13908693 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6730014480 con 0x7f67380d0f50 2024-04-22T07:03:59.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f6734ff9640 1 -- 172.21.15.92:0/13908693 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730014870 con 0x7f67380d0f50 2024-04-22T07:03:59.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f673d843640 1 -- 172.21.15.92:0/13908693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67381f13e0 con 0x7f67380d0f50 2024-04-22T07:03:59.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.944+0000 7f673d843640 1 -- 172.21.15.92:0/13908693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67380cfff0 con 0x7f67380d0f50 2024-04-22T07:03:59.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.945+0000 7f673d843640 1 -- 172.21.15.92:0/13908693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6738069930 con 0x7f67380d0f50 2024-04-22T07:03:59.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.946+0000 7f6734ff9640 1 -- 172.21.15.92:0/13908693 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f6730005d10 con 0x7f67380d0f50 2024-04-22T07:03:59.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.946+0000 7f6734ff9640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f671806c860 0x7f671806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.946+0000 7f6734ff9640 1 -- 172.21.15.92:0/13908693 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4857+0+0 (secure 0 0 0) 0x7f673008c3f0 con 0x7f67380d0f50 2024-04-22T07:03:59.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.946+0000 7f6736ffd640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f671806c860 0x7f671806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.946+0000 7f6736ffd640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f671806c860 0x7f671806ed20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6728004ea0 tx=0x7f6728009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:59.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:03:59.951+0000 7f6734ff9640 1 -- 172.21.15.92:0/13908693 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6730022030 con 0x7f67380d0f50 2024-04-22T07:04:00.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.131+0000 7f673d843640 1 -- 172.21.15.92:0/13908693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67380c6360 con 0x7f67380d0f50 2024-04-22T07:04:00.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.132+0000 7f6734ff9640 1 -- 172.21.15.92:0/13908693 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12836 (secure 0 0 0) 0x7f6730010040 con 0x7f67380d0f50 2024-04-22T07:04:00.131 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:00.131 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":16,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:03:59.964796+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:00.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.133+0000 7f673d843640 1 -- 172.21.15.92:0/13908693 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f671806c860 msgr2=0x7f671806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.134+0000 7f673d843640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f671806c860 0x7f671806ed20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6728004ea0 tx=0x7f6728009040 comp rx=0 tx=0).stop 2024-04-22T07:04:00.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.134+0000 7f673d843640 1 -- 172.21.15.92:0/13908693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67380d0f50 msgr2=0x7f67380cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.134+0000 7f673d843640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67380d0f50 0x7f67380cf0b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f673000efc0 tx=0x7f673000c490 comp rx=0 tx=0).stop 2024-04-22T07:04:00.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.134+0000 7f673d843640 1 -- 172.21.15.92:0/13908693 shutdown_connections 2024-04-22T07:04:00.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.134+0000 7f673d843640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f671806c860 0x7f671806ed20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.134+0000 7f673d843640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67380d0f50 0x7f67380cf0b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.134+0000 7f673d843640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67380d0b70 0x7f67380ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.134+0000 7f673d843640 1 --2- 172.21.15.92:0/13908693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67380cb120 0x7f67380c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.134+0000 7f673d843640 1 -- 172.21.15.92:0/13908693 >> 172.21.15.92:0/13908693 conn(0x7f67380736b0 msgr2=0x7f67381590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:00.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.134+0000 7f673d843640 1 -- 172.21.15.92:0/13908693 shutdown_connections 2024-04-22T07:04:00.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.134+0000 7f673d843640 1 -- 172.21.15.92:0/13908693 wait complete. 2024-04-22T07:04:00.143 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:03:57.947601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:00.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:00.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.245+0000 7fede4c55640 1 Processor -- start 2024-04-22T07:04:00.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.245+0000 7fede4c55640 1 -- start start 2024-04-22T07:04:00.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.246+0000 7fede4c55640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fede0151c00 0x7fede00c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.246+0000 7fede4c55640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fede006eed0 0x7fede00c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.246+0000 7fede4c55640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede00cd100 con 0x7fede00c6c20 2024-04-22T07:04:00.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.246+0000 7fede4c55640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fede00c7580 con 0x7fede0151c00 2024-04-22T07:04:00.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.246+0000 7fede4c55640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fede00c72a0 con 0x7fede006eed0 2024-04-22T07:04:00.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.246+0000 7fedde59c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fede0151c00 0x7fede00c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.246+0000 7fedd7fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fede006eed0 0x7fede00c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.246+0000 7fedd7fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fede006eed0 0x7fede00c5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:43932/0 (socket says 172.21.15.92:43932) 2024-04-22T07:04:00.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.246+0000 7fedde59c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fede0151c00 0x7fede00c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:51966/0 (socket says 172.21.15.92:51966) 2024-04-22T07:04:00.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.247+0000 7fedd7fff640 1 -- 172.21.15.92:0/3413826989 learned_addr learned my addr 172.21.15.92:0/3413826989 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:00.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.246+0000 7fedded9d640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fede00c6c20 0x7fede015af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:51572/0 (socket says 172.21.15.92:51572) 2024-04-22T07:04:00.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.247+0000 7fedd7fff640 1 -- 172.21.15.92:0/3413826989 >> v2:172.21.15.145:3300/0 conn(0x7fede0151c00 msgr2=0x7fede00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:00.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.247+0000 7fedd7fff640 1 --2- 172.21.15.92:0/3413826989 >> v2:172.21.15.145:3300/0 conn(0x7fede0151c00 0x7fede00c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.247+0000 7fedd7fff640 1 -- 172.21.15.92:0/3413826989 >> v1:172.21.15.145:6789/0 conn(0x7fede00c6c20 legacy=0x7fede015af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.247+0000 7fedd7fff640 1 -- 172.21.15.92:0/3413826989 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede00c87d0 con 0x7fede006eed0 2024-04-22T07:04:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.247+0000 7fedde59c640 1 --2- 172.21.15.92:0/3413826989 >> v2:172.21.15.145:3300/0 conn(0x7fede0151c00 0x7fede00c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.248+0000 7fedd7fff640 1 --2- 172.21.15.92:0/3413826989 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fede006eed0 0x7fede00c5880 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009870 tx=0x7fedc802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd71af186f78f618 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.248+0000 7fedddd9b640 1 -- 172.21.15.92:0/3413826989 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8034070 con 0x7fede006eed0 2024-04-22T07:04:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.248+0000 7fedddd9b640 1 -- 172.21.15.92:0/3413826989 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc8030840 con 0x7fede006eed0 2024-04-22T07:04:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.248+0000 7fedddd9b640 1 -- 172.21.15.92:0/3413826989 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8030c10 con 0x7fede006eed0 2024-04-22T07:04:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.248+0000 7fede4c55640 1 -- 172.21.15.92:0/3413826989 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fede006eed0 msgr2=0x7fede00c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.248+0000 7fede4c55640 1 --2- 172.21.15.92:0/3413826989 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fede006eed0 0x7fede00c5880 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009870 tx=0x7fedc802b980 comp rx=0 tx=0).stop 2024-04-22T07:04:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.248+0000 7fede4c55640 1 -- 172.21.15.92:0/3413826989 shutdown_connections 2024-04-22T07:04:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.248+0000 7fede4c55640 1 --2- 172.21.15.92:0/3413826989 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fede006eed0 0x7fede00c5880 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.248+0000 7fede4c55640 1 --2- 172.21.15.92:0/3413826989 >> v2:172.21.15.145:3300/0 conn(0x7fede0151c00 0x7fede00c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.248+0000 7fede4c55640 1 -- 172.21.15.92:0/3413826989 >> 172.21.15.92:0/3413826989 conn(0x7fede00cc220 msgr2=0x7fede00cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.248+0000 7fede4c55640 1 -- 172.21.15.92:0/3413826989 shutdown_connections 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.248+0000 7fede4c55640 1 -- 172.21.15.92:0/3413826989 wait complete. 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.249+0000 7fede4c55640 1 Processor -- start 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.250+0000 7fede4c55640 1 -- start start 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.250+0000 7fede4c55640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fede006eed0 0x7fede0136f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.250+0000 7fede4c55640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fede00c6c20 0x7fede00c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.250+0000 7fede4c55640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fede0151c00 0x7fede00c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.250+0000 7fede4c55640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fede0151fe0 con 0x7fede006eed0 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.250+0000 7fede4c55640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fede00c37c0 con 0x7fede0151c00 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.250+0000 7fede4c55640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fede015b660 con 0x7fede00c6c20 2024-04-22T07:04:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.250+0000 7fedde59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fede006eed0 0x7fede0136f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.250+0000 7fedde59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fede006eed0 0x7fede0136f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44962/0 (socket says 172.21.15.92:44962) 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.250+0000 7fedde59c640 1 -- 172.21.15.92:0/1973588722 learned_addr learned my addr 172.21.15.92:0/1973588722 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.250+0000 7fedd7fff640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fede00c6c20 0x7fede00c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.250+0000 7fedded9d640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fede0151c00 0x7fede00c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.251+0000 7fedde59c640 1 -- 172.21.15.92:0/1973588722 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fede00c6c20 msgr2=0x7fede00c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.251+0000 7fedde59c640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fede00c6c20 0x7fede00c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.251+0000 7fedde59c640 1 -- 172.21.15.92:0/1973588722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fede0151c00 msgr2=0x7fede00c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.251+0000 7fedde59c640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fede0151c00 0x7fede00c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.251+0000 7fedde59c640 1 -- 172.21.15.92:0/1973588722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede00cbc10 con 0x7fede006eed0 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.251+0000 7fedded9d640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fede0151c00 0x7fede00c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.251+0000 7fedde59c640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fede006eed0 0x7fede0136f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fedd000e990 tx=0x7fedd000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.251+0000 7fedd6ffd640 1 -- 172.21.15.92:0/1973588722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0017070 con 0x7fede006eed0 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.251+0000 7fedd6ffd640 1 -- 172.21.15.92:0/1973588722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd000c9e0 con 0x7fede006eed0 2024-04-22T07:04:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.251+0000 7fede4c55640 1 -- 172.21.15.92:0/1973588722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede00cbeb0 con 0x7fede006eed0 2024-04-22T07:04:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.251+0000 7fedd6ffd640 1 -- 172.21.15.92:0/1973588722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd000cdb0 con 0x7fede006eed0 2024-04-22T07:04:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.251+0000 7fede4c55640 1 -- 172.21.15.92:0/1973588722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede014c2b0 con 0x7fede006eed0 2024-04-22T07:04:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.252+0000 7fede4c55640 1 -- 172.21.15.92:0/1973588722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede0069930 con 0x7fede006eed0 2024-04-22T07:04:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.252+0000 7fedd6ffd640 1 -- 172.21.15.92:0/1973588722 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fedd001a070 con 0x7fede006eed0 2024-04-22T07:04:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.253+0000 7fedd6ffd640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fedc006c890 0x7fedc006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.253+0000 7fedd6ffd640 1 -- 172.21.15.92:0/1973588722 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fedd008b510 con 0x7fede006eed0 2024-04-22T07:04:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.253+0000 7fedd7fff640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fedc006c890 0x7fedc006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.253+0000 7fedd7fff640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fedc006c890 0x7fedc006ed50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fedc8036000 tx=0x7fedc803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.257+0000 7fedd6ffd640 1 -- 172.21.15.92:0/1973588722 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedd0059e00 con 0x7fede006eed0 2024-04-22T07:04:00.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.430+0000 7fede4c55640 1 -- 172.21.15.92:0/1973588722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fede00c62c0 con 0x7fede006eed0 2024-04-22T07:04:00.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.431+0000 7fedd6ffd640 1 -- 172.21.15.92:0/1973588722 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fedd008b070 con 0x7fede006eed0 2024-04-22T07:04:00.429 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-04-22T07:04:00.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.432+0000 7fede4c55640 1 -- 172.21.15.92:0/1973588722 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fedc006c890 msgr2=0x7fedc006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.432+0000 7fede4c55640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fedc006c890 0x7fedc006ed50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fedc8036000 tx=0x7fedc803b040 comp rx=0 tx=0).stop 2024-04-22T07:04:00.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.433+0000 7fede4c55640 1 -- 172.21.15.92:0/1973588722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fede006eed0 msgr2=0x7fede0136f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.433+0000 7fede4c55640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fede006eed0 0x7fede0136f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fedd000e990 tx=0x7fedd000ee50 comp rx=0 tx=0).stop 2024-04-22T07:04:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.433+0000 7fede4c55640 1 -- 172.21.15.92:0/1973588722 shutdown_connections 2024-04-22T07:04:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.433+0000 7fede4c55640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fedc006c890 0x7fedc006ed50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.433+0000 7fede4c55640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fede0151c00 0x7fede00c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.433+0000 7fede4c55640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fede00c6c20 0x7fede00c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.433+0000 7fede4c55640 1 --2- 172.21.15.92:0/1973588722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fede006eed0 0x7fede0136f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.433+0000 7fede4c55640 1 -- 172.21.15.92:0/1973588722 >> 172.21.15.92:0/1973588722 conn(0x7fede00cc220 msgr2=0x7fede014f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.433+0000 7fede4c55640 1 -- 172.21.15.92:0/1973588722 shutdown_connections 2024-04-22T07:04:00.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.433+0000 7fede4c55640 1 -- 172.21.15.92:0/1973588722 wait complete. 2024-04-22T07:04:00.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-22T07:04:00.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.548+0000 7f124cba4640 1 Processor -- start 2024-04-22T07:04:00.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.548+0000 7f124cba4640 1 -- start start 2024-04-22T07:04:00.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.549+0000 7f124cba4640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1248152cf0 0x7f12480c5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.549+0000 7f124cba4640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12480c8ab0 con 0x7f12480c6b40 2024-04-22T07:04:00.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.549+0000 7f124cba4640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12480c8d90 con 0x7f12480c8220 2024-04-22T07:04:00.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.549+0000 7f124cba4640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f12480c9070 con 0x7f1248152cf0 2024-04-22T07:04:00.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.549+0000 7f124659c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1248152cf0 0x7f12480c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.549+0000 7f124659c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1248152cf0 0x7f12480c5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44968/0 (socket says 172.21.15.92:44968) 2024-04-22T07:04:00.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.549+0000 7f124659c640 1 -- 172.21.15.92:0/2363379476 learned_addr learned my addr 172.21.15.92:0/2363379476 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:00.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.549+0000 7f124659c640 1 -- 172.21.15.92:0/2363379476 >> v1:172.21.15.145:6789/0 conn(0x7f12480c8220 legacy=0x7f12480c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:00.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.549+0000 7f124659c640 1 -- 172.21.15.92:0/2363379476 >> v1:172.21.15.92:6789/0 conn(0x7f12480c6b40 legacy=0x7f124815c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.549+0000 7f124659c640 1 -- 172.21.15.92:0/2363379476 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12480c7410 con 0x7f1248152cf0 2024-04-22T07:04:00.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.549+0000 7f1245d9b640 1 -- 172.21.15.92:0/2363379476 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1528904219 0 0) 0x7f12480c8ab0 con 0x7f12480c6b40 2024-04-22T07:04:00.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.550+0000 7f124659c640 1 --2- 172.21.15.92:0/2363379476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1248152cf0 0x7f12480c5190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1238009870 tx=0x7f123802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c5f3f20d0e1362 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:00.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.550+0000 7f1245d9b640 1 -- 172.21.15.92:0/2363379476 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123802f840 con 0x7f1248152cf0 2024-04-22T07:04:00.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.550+0000 7f1245d9b640 1 -- 172.21.15.92:0/2363379476 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f123802f9e0 con 0x7f1248152cf0 2024-04-22T07:04:00.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.550+0000 7f1245d9b640 1 -- 172.21.15.92:0/2363379476 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123802fdb0 con 0x7f1248152cf0 2024-04-22T07:04:00.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.550+0000 7f124cba4640 1 -- 172.21.15.92:0/2363379476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1248152cf0 msgr2=0x7f12480c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.550+0000 7f124cba4640 1 --2- 172.21.15.92:0/2363379476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1248152cf0 0x7f12480c5190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1238009870 tx=0x7f123802ba20 comp rx=0 tx=0).stop 2024-04-22T07:04:00.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.551+0000 7f124cba4640 1 -- 172.21.15.92:0/2363379476 shutdown_connections 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.551+0000 7f124cba4640 1 --2- 172.21.15.92:0/2363379476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1248152cf0 0x7f12480c5190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.551+0000 7f124cba4640 1 -- 172.21.15.92:0/2363379476 >> 172.21.15.92:0/2363379476 conn(0x7f12480cc970 msgr2=0x7f12480ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.551+0000 7f124cba4640 1 -- 172.21.15.92:0/2363379476 shutdown_connections 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.551+0000 7f124cba4640 1 -- 172.21.15.92:0/2363379476 wait complete. 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.552+0000 7f124cba4640 1 Processor -- start 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.552+0000 7f124cba4640 1 -- start start 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.552+0000 7f124cba4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f12480c6b40 0x7f12481e2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.552+0000 7f124cba4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12480c8220 0x7f12481565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.552+0000 7f124cba4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1248152cf0 0x7f12481504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.552+0000 7f124cba4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f12480c9b10 con 0x7f1248152cf0 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.552+0000 7f124cba4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f12480c79d0 con 0x7f12480c8220 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.552+0000 7f124cba4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f12480c9c90 con 0x7f12480c6b40 2024-04-22T07:04:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f1246d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1248152cf0 0x7f12481504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f1246d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1248152cf0 0x7f12481504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44976/0 (socket says 172.21.15.92:44976) 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f1246d9d640 1 -- 172.21.15.92:0/2454297960 learned_addr learned my addr 172.21.15.92:0/2454297960 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f124659c640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f12480c6b40 0x7f12481e2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f123ffff640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12480c8220 0x7f12481565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f1246d9d640 1 -- 172.21.15.92:0/2454297960 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f12480c6b40 msgr2=0x7f12481e2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f1246d9d640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f12480c6b40 0x7f12481e2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f1246d9d640 1 -- 172.21.15.92:0/2454297960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12480c8220 msgr2=0x7f12481565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f1246d9d640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12480c8220 0x7f12481565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f1246d9d640 1 -- 172.21.15.92:0/2454297960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1248159690 con 0x7f1248152cf0 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f1246d9d640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1248152cf0 0x7f12481504f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f124000c790 tx=0x7f124000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f123effd640 1 -- 172.21.15.92:0/2454297960 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240003da0 con 0x7f1248152cf0 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f123effd640 1 -- 172.21.15.92:0/2454297960 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1240005070 con 0x7f1248152cf0 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f124cba4640 1 -- 172.21.15.92:0/2454297960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12481eec00 con 0x7f1248152cf0 2024-04-22T07:04:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f123effd640 1 -- 172.21.15.92:0/2454297960 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240005460 con 0x7f1248152cf0 2024-04-22T07:04:00.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.553+0000 7f124cba4640 1 -- 172.21.15.92:0/2454297960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12481ef1c0 con 0x7f1248152cf0 2024-04-22T07:04:00.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.554+0000 7f124cba4640 1 -- 172.21.15.92:0/2454297960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1248069510 con 0x7f1248152cf0 2024-04-22T07:04:00.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.554+0000 7f123effd640 1 -- 172.21.15.92:0/2454297960 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f1240005600 con 0x7f1248152cf0 2024-04-22T07:04:00.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.555+0000 7f123effd640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f122806c860 0x7f122806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.555+0000 7f123effd640 1 -- 172.21.15.92:0/2454297960 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f124008b4d0 con 0x7f1248152cf0 2024-04-22T07:04:00.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.555+0000 7f124659c640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f122806c860 0x7f122806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.555+0000 7f124659c640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f122806c860 0x7f122806ed20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1238036000 tx=0x7f1238008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:00.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.559+0000 7f123effd640 1 -- 172.21.15.92:0/2454297960 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1248069510 con 0x7f1248152cf0 2024-04-22T07:04:00.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.735+0000 7f124cba4640 1 -- 172.21.15.92:0/2454297960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1248069930 con 0x7f1248152cf0 2024-04-22T07:04:00.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.735+0000 7f123effd640 1 -- 172.21.15.92:0/2454297960 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f12400596e0 con 0x7f1248152cf0 2024-04-22T07:04:00.739 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:00.740 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2024-04-22T07:04:00.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-22T07:04:00.740 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2024-04-22T07:04:00.740 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2024-04-22T07:04:00.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:04:00.740 INFO:teuthology.orchestra.run.smithi092.stdout:] 2024-04-22T07:04:00.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.737+0000 7f124cba4640 1 -- 172.21.15.92:0/2454297960 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f122806c860 msgr2=0x7f122806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.737+0000 7f124cba4640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f122806c860 0x7f122806ed20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1238036000 tx=0x7f1238008030 comp rx=0 tx=0).stop 2024-04-22T07:04:00.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.737+0000 7f124cba4640 1 -- 172.21.15.92:0/2454297960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1248152cf0 msgr2=0x7f12481504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.737+0000 7f124cba4640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1248152cf0 0x7f12481504f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f124000c790 tx=0x7f124000cc50 comp rx=0 tx=0).stop 2024-04-22T07:04:00.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.737+0000 7f124cba4640 1 -- 172.21.15.92:0/2454297960 shutdown_connections 2024-04-22T07:04:00.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.737+0000 7f124cba4640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f122806c860 0x7f122806ed20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.737+0000 7f124cba4640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1248152cf0 0x7f12481504f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.737+0000 7f124cba4640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12480c8220 0x7f12481565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.737+0000 7f124cba4640 1 --2- 172.21.15.92:0/2454297960 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f12480c6b40 0x7f12481e2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.737+0000 7f124cba4640 1 -- 172.21.15.92:0/2454297960 >> 172.21.15.92:0/2454297960 conn(0x7f12480cc970 msgr2=0x7f12480cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:00.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.737+0000 7f124cba4640 1 -- 172.21.15.92:0/2454297960 shutdown_connections 2024-04-22T07:04:00.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.737+0000 7f124cba4640 1 -- 172.21.15.92:0/2454297960 wait complete. 2024-04-22T07:04:00.756 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-22T07:04:00.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-22T07:04:00.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.864+0000 7f6052077640 1 Processor -- start 2024-04-22T07:04:00.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.864+0000 7f6052077640 1 -- start start 2024-04-22T07:04:00.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.865+0000 7f6052077640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f604c15ab20 0x7f604c154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.865+0000 7f6052077640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f604c073cd0 con 0x7f604c0c74a0 2024-04-22T07:04:00.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.865+0000 7f6052077640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f604c0ca490 con 0x7f604c154ad0 2024-04-22T07:04:00.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.865+0000 7f6052077640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f604c073f90 con 0x7f604c15ab20 2024-04-22T07:04:00.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.865+0000 7f604affd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f604c0c74a0 0x7f604c163470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39030/0 (socket says 172.21.15.92:39030) 2024-04-22T07:04:00.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.865+0000 7f604affd640 1 -- 172.21.15.92:0/1066407073 learned_addr learned my addr 172.21.15.92:0/1066407073 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:00.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604bfff640 1 --2- 172.21.15.92:0/1066407073 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f604c15ab20 0x7f604c154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4032587262 0 0) 0x7f604c073cd0 con 0x7f604c0c74a0 2024-04-22T07:04:00.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6034003520 con 0x7f604c0c74a0 2024-04-22T07:04:00.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1846639852 0 0) 0x7f6034003520 con 0x7f604c0c74a0 2024-04-22T07:04:00.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f604c15ab20 msgr2=0x7f604c154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:00.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 --2- 172.21.15.92:0/1066407073 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f604c15ab20 0x7f604c154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 >> v1:172.21.15.145:6789/0 conn(0x7f604c154ad0 legacy=0x7f604c06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c0ca770 con 0x7f604c0c74a0 2024-04-22T07:04:00.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f6040001670 con 0x7f604c0c74a0 2024-04-22T07:04:00.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 >> v1:172.21.15.92:6789/0 conn(0x7f604c0c74a0 legacy=0x7f604c163470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 --2- 172.21.15.92:0/1066407073 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f604c15ab20 0x7f6034003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 --2- 172.21.15.92:0/1066407073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6034003f80 0x7f6034004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 --2- 172.21.15.92:0/1066407073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6034004d40 0x7f60340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f604c073f90 con 0x7f6034004d40 2024-04-22T07:04:00.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f603400bb80 con 0x7f6034003f80 2024-04-22T07:04:00.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f603400bd80 con 0x7f604c15ab20 2024-04-22T07:04:00.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604bfff640 1 --2- 172.21.15.92:0/1066407073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6034004d40 0x7f60340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604b7fe640 1 --2- 172.21.15.92:0/1066407073 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f604c15ab20 0x7f6034003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604affd640 1 --2- 172.21.15.92:0/1066407073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6034003f80 0x7f6034004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604bfff640 1 -- 172.21.15.92:0/1066407073 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f604c15ab20 msgr2=0x7f6034003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:00.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604bfff640 1 --2- 172.21.15.92:0/1066407073 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f604c15ab20 0x7f6034003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604bfff640 1 -- 172.21.15.92:0/1066407073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6034003f80 msgr2=0x7f6034004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:00.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604bfff640 1 --2- 172.21.15.92:0/1066407073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6034003f80 0x7f6034004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.866+0000 7f604bfff640 1 -- 172.21.15.92:0/1066407073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6034003520 con 0x7f6034004d40 2024-04-22T07:04:00.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.867+0000 7f604bfff640 1 --2- 172.21.15.92:0/1066407073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6034004d40 0x7f60340051c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6044009fc0 tx=0x7f6034005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5618d6fd9c9850 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:00.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.867+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044039070 con 0x7f6034004d40 2024-04-22T07:04:00.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.867+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604402cc20 con 0x7f6034004d40 2024-04-22T07:04:00.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.867+0000 7f604a7fc640 1 -- 172.21.15.92:0/1066407073 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044009c10 con 0x7f6034004d40 2024-04-22T07:04:00.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.868+0000 7f6052077640 1 -- 172.21.15.92:0/1066407073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6034004d40 msgr2=0x7f60340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.868+0000 7f6052077640 1 --2- 172.21.15.92:0/1066407073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6034004d40 0x7f60340051c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6044009fc0 tx=0x7f6034005830 comp rx=0 tx=0).stop 2024-04-22T07:04:00.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.868+0000 7f604bfff640 1 -- 172.21.15.92:0/1066407073 reap_dead start 2024-04-22T07:04:00.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.868+0000 7f6052077640 1 -- 172.21.15.92:0/1066407073 shutdown_connections 2024-04-22T07:04:00.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.868+0000 7f6052077640 1 -- 172.21.15.92:0/1066407073 >> 172.21.15.92:0/1066407073 conn(0x7f604c070950 msgr2=0x7f604c070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:00.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.868+0000 7f6052077640 1 -- 172.21.15.92:0/1066407073 shutdown_connections 2024-04-22T07:04:00.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.868+0000 7f6052077640 1 -- 172.21.15.92:0/1066407073 wait complete. 2024-04-22T07:04:00.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.869+0000 7f6052077640 1 Processor -- start 2024-04-22T07:04:00.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.869+0000 7f6052077640 1 -- start start 2024-04-22T07:04:00.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f6052077640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6034004d40 0x7f604c152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f6052077640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f604c152180 0x7f604c0cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f6052077640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f604c1e45c0 0x7f604c0cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f6052077640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f604c0738e0 con 0x7f604c152180 2024-04-22T07:04:00.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f6052077640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f604c0c89a0 con 0x7f604c1e45c0 2024-04-22T07:04:00.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f6052077640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f604c073a60 con 0x7f6034004d40 2024-04-22T07:04:00.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f604affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f604c152180 0x7f604c0cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f604affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f604c152180 0x7f604c0cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45004/0 (socket says 172.21.15.92:45004) 2024-04-22T07:04:00.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f604affd640 1 -- 172.21.15.92:0/3117277368 learned_addr learned my addr 172.21.15.92:0/3117277368 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:00.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f604affd640 1 -- 172.21.15.92:0/3117277368 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6034004d40 msgr2=0x7f604c152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f604b7fe640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6034004d40 0x7f604c152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f604bfff640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f604c1e45c0 0x7f604c0cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f604affd640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6034004d40 0x7f604c152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f604affd640 1 -- 172.21.15.92:0/3117277368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f604c1e45c0 msgr2=0x7f604c0cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:00.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f604affd640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f604c1e45c0 0x7f604c0cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:00.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f604affd640 1 -- 172.21.15.92:0/3117277368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c1f1410 con 0x7f604c152180 2024-04-22T07:04:00.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f604affd640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f604c152180 0x7f604c0cc000 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60400072c0 tx=0x7f6040007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:00.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f6033fff640 1 -- 172.21.15.92:0/3117277368 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604000f960 con 0x7f604c152180 2024-04-22T07:04:00.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f6033fff640 1 -- 172.21.15.92:0/3117277368 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6040018070 con 0x7f604c152180 2024-04-22T07:04:00.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f6052077640 1 -- 172.21.15.92:0/3117277368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604c1f16d0 con 0x7f604c152180 2024-04-22T07:04:00.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f6033fff640 1 -- 172.21.15.92:0/3117277368 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604000fd40 con 0x7f604c152180 2024-04-22T07:04:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.870+0000 7f6052077640 1 -- 172.21.15.92:0/3117277368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604c0cb520 con 0x7f604c152180 2024-04-22T07:04:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.871+0000 7f6052077640 1 -- 172.21.15.92:0/3117277368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f604c069ab0 con 0x7f604c152180 2024-04-22T07:04:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.872+0000 7f6033fff640 1 -- 172.21.15.92:0/3117277368 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f604001d460 con 0x7f604c152180 2024-04-22T07:04:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.872+0000 7f6033fff640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f603806c8f0 0x7f603806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.872+0000 7f604b7fe640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f603806c8f0 0x7f603806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.872+0000 7f6033fff640 1 -- 172.21.15.92:0/3117277368 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f604008b9e0 con 0x7f604c152180 2024-04-22T07:04:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.873+0000 7f604b7fe640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f603806c8f0 0x7f603806edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f603c0055b0 tx=0x7f603c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:00.876+0000 7f6033fff640 1 -- 172.21.15.92:0/3117277368 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f604005a160 con 0x7f604c152180 2024-04-22T07:04:01.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.054+0000 7f6052077640 1 -- 172.21.15.92:0/3117277368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f604c0692b0 con 0x7f604c152180 2024-04-22T07:04:01.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.764+0000 7f6033fff640 1 -- 172.21.15.92:0/3117277368 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f6040059a60 con 0x7f604c152180 2024-04-22T07:04:01.763 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2024-04-22T07:04:01.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.767+0000 7f6052077640 1 -- 172.21.15.92:0/3117277368 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f603806c8f0 msgr2=0x7f603806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:01.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.767+0000 7f6052077640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f603806c8f0 0x7f603806edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f603c0055b0 tx=0x7f603c009040 comp rx=0 tx=0).stop 2024-04-22T07:04:01.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.767+0000 7f6052077640 1 -- 172.21.15.92:0/3117277368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f604c152180 msgr2=0x7f604c0cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:01.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.767+0000 7f6052077640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f604c152180 0x7f604c0cc000 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60400072c0 tx=0x7f6040007780 comp rx=0 tx=0).stop 2024-04-22T07:04:01.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.767+0000 7f6052077640 1 -- 172.21.15.92:0/3117277368 shutdown_connections 2024-04-22T07:04:01.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.767+0000 7f6052077640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f603806c8f0 0x7f603806edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:01.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.767+0000 7f6052077640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f604c1e45c0 0x7f604c0cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:01.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.767+0000 7f6052077640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f604c152180 0x7f604c0cc000 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:01.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.767+0000 7f6052077640 1 --2- 172.21.15.92:0/3117277368 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6034004d40 0x7f604c152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:01.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.767+0000 7f6052077640 1 -- 172.21.15.92:0/3117277368 >> 172.21.15.92:0/3117277368 conn(0x7f604c070950 msgr2=0x7f604c072d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:01.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.768+0000 7f6052077640 1 -- 172.21.15.92:0/3117277368 shutdown_connections 2024-04-22T07:04:01.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.768+0000 7f6052077640 1 -- 172.21.15.92:0/3117277368 wait complete. 2024-04-22T07:04:01.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-22T07:04:01.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.890+0000 7fb500c5c640 1 Processor -- start 2024-04-22T07:04:01.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.890+0000 7fb500c5c640 1 -- start start 2024-04-22T07:04:01.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.891+0000 7fb500c5c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb4fc06d770 0x7fb4fc06f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:01.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.891+0000 7fb500c5c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4fc0b0eb0 con 0x7fb4fc158390 2024-04-22T07:04:01.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.891+0000 7fb500c5c640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4fc0ca6e0 con 0x7fb4fc0c9730 2024-04-22T07:04:01.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.891+0000 7fb500c5c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fb4fc158770 con 0x7fb4fc06d770 2024-04-22T07:04:01.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.891+0000 7fb4f9d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb4fc06d770 0x7fb4fc06f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:01.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.891+0000 7fb4f9d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb4fc06d770 0x7fb4fc06f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45006/0 (socket says 172.21.15.92:45006) 2024-04-22T07:04:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.891+0000 7fb4fa59c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb4fc158390 0x7fb4fc06bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39034/0 (socket says 172.21.15.92:39034) 2024-04-22T07:04:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.891+0000 7fb4f9d9b640 1 -- 172.21.15.92:0/898148814 learned_addr learned my addr 172.21.15.92:0/898148814 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.891+0000 7fb4f9d9b640 1 -- 172.21.15.92:0/898148814 >> v1:172.21.15.145:6789/0 conn(0x7fb4fc0c9730 legacy=0x7fb4fc1655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.891+0000 7fb4f9d9b640 1 -- 172.21.15.92:0/898148814 >> v1:172.21.15.92:6789/0 conn(0x7fb4fc158390 legacy=0x7fb4fc06bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.891+0000 7fb4f959a640 1 -- 172.21.15.92:0/898148814 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211219637 0 0) 0x7fb4fc0b0eb0 con 0x7fb4fc158390 2024-04-22T07:04:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.891+0000 7fb4f9d9b640 1 -- 172.21.15.92:0/898148814 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4fc0ca9c0 con 0x7fb4fc06d770 2024-04-22T07:04:01.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.892+0000 7fb4f9d9b640 1 --2- 172.21.15.92:0/898148814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4fc06d770 0x7fb4fc06f5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009870 tx=0x7fb4f002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea97750750ae2bca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:01.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.892+0000 7fb4f959a640 1 -- 172.21.15.92:0/898148814 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f002f840 con 0x7fb4fc06d770 2024-04-22T07:04:01.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.892+0000 7fb4f959a640 1 -- 172.21.15.92:0/898148814 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f002f9e0 con 0x7fb4fc06d770 2024-04-22T07:04:01.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.892+0000 7fb4f959a640 1 -- 172.21.15.92:0/898148814 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f002fdb0 con 0x7fb4fc06d770 2024-04-22T07:04:01.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.893+0000 7fb500c5c640 1 -- 172.21.15.92:0/898148814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4fc06d770 msgr2=0x7fb4fc06f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:01.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.893+0000 7fb500c5c640 1 --2- 172.21.15.92:0/898148814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4fc06d770 0x7fb4fc06f5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009870 tx=0x7fb4f002ba20 comp rx=0 tx=0).stop 2024-04-22T07:04:01.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.893+0000 7fb500c5c640 1 -- 172.21.15.92:0/898148814 shutdown_connections 2024-04-22T07:04:01.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.893+0000 7fb500c5c640 1 --2- 172.21.15.92:0/898148814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4fc06d770 0x7fb4fc06f5a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:01.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.893+0000 7fb500c5c640 1 -- 172.21.15.92:0/898148814 >> 172.21.15.92:0/898148814 conn(0x7fb4fc0676b0 msgr2=0x7fb4fc06db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:01.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.893+0000 7fb500c5c640 1 -- 172.21.15.92:0/898148814 shutdown_connections 2024-04-22T07:04:01.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.893+0000 7fb500c5c640 1 -- 172.21.15.92:0/898148814 wait complete. 2024-04-22T07:04:01.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.894+0000 7fb500c5c640 1 Processor -- start 2024-04-22T07:04:01.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.894+0000 7fb500c5c640 1 -- start start 2024-04-22T07:04:01.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb500c5c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4fc06d770 0x7fb4fc0b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:01.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb500c5c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb4fc0c9730 0x7fb4fc1f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb500c5c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb4fc158390 0x7fb4fc0afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb500c5c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc0c9d20 con 0x7fb4fc06d770 2024-04-22T07:04:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb500c5c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc0c9ba0 con 0x7fb4fc158390 2024-04-22T07:04:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb500c5c640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb4fc0ce3e0 con 0x7fb4fc0c9730 2024-04-22T07:04:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4fa59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4fc06d770 0x7fb4fc0b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4fa59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4fc06d770 0x7fb4fc0b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45010/0 (socket says 172.21.15.92:45010) 2024-04-22T07:04:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4fa59c640 1 -- 172.21.15.92:0/3061245997 learned_addr learned my addr 172.21.15.92:0/3061245997 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4f9d9b640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb4fc0c9730 0x7fb4fc1f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4fad9d640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb4fc158390 0x7fb4fc0afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4fa59c640 1 -- 172.21.15.92:0/3061245997 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb4fc0c9730 msgr2=0x7fb4fc1f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4fa59c640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb4fc0c9730 0x7fb4fc1f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4fa59c640 1 -- 172.21.15.92:0/3061245997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb4fc158390 msgr2=0x7fb4fc0afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4fa59c640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb4fc158390 0x7fb4fc0afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4fa59c640 1 -- 172.21.15.92:0/3061245997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4fc0b14d0 con 0x7fb4fc06d770 2024-04-22T07:04:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4fad9d640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb4fc158390 0x7fb4fc0afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4fa59c640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4fc06d770 0x7fb4fc0b2250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00c7c0 tx=0x7fb4ec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4eaffd640 1 -- 172.21.15.92:0/3061245997 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec018070 con 0x7fb4fc06d770 2024-04-22T07:04:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4eaffd640 1 -- 172.21.15.92:0/3061245997 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4ec00a5e0 con 0x7fb4fc06d770 2024-04-22T07:04:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb500c5c640 1 -- 172.21.15.92:0/3061245997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4fc1f1a60 con 0x7fb4fc06d770 2024-04-22T07:04:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb4eaffd640 1 -- 172.21.15.92:0/3061245997 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec00a9d0 con 0x7fb4fc06d770 2024-04-22T07:04:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.895+0000 7fb500c5c640 1 -- 172.21.15.92:0/3061245997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4fc0b1a50 con 0x7fb4fc06d770 2024-04-22T07:04:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.897+0000 7fb500c5c640 1 -- 172.21.15.92:0/3061245997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4fc069930 con 0x7fb4fc06d770 2024-04-22T07:04:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.897+0000 7fb4eaffd640 1 -- 172.21.15.92:0/3061245997 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4ec01b070 con 0x7fb4fc06d770 2024-04-22T07:04:01.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.897+0000 7fb4eaffd640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb4dc06c8f0 0x7fb4dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:01.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.897+0000 7fb4f9d9b640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb4dc06c8f0 0x7fb4dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:01.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.897+0000 7fb4eaffd640 1 -- 172.21.15.92:0/3061245997 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fb4ec08af50 con 0x7fb4fc06d770 2024-04-22T07:04:01.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.898+0000 7fb4f9d9b640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb4dc06c8f0 0x7fb4dc06edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0036000 tx=0x7fb4f0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:01.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:01.902+0000 7fb4eaffd640 1 -- 172.21.15.92:0/3061245997 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4ec058e80 con 0x7fb4fc06d770 2024-04-22T07:04:02.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.081+0000 7fb500c5c640 1 -- 172.21.15.92:0/3061245997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb4fc06db50 con 0x7fb4fc06d770 2024-04-22T07:04:02.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.772+0000 7fb4eaffd640 1 -- 172.21.15.92:0/3061245997 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7fb4ec05d180 con 0x7fb4fc06d770 2024-04-22T07:04:02.770 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2024-04-22T07:04:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.774+0000 7fb500c5c640 1 -- 172.21.15.92:0/3061245997 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb4dc06c8f0 msgr2=0x7fb4dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.774+0000 7fb500c5c640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb4dc06c8f0 0x7fb4dc06edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0036000 tx=0x7fb4f0008030 comp rx=0 tx=0).stop 2024-04-22T07:04:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.774+0000 7fb500c5c640 1 -- 172.21.15.92:0/3061245997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4fc06d770 msgr2=0x7fb4fc0b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.774+0000 7fb500c5c640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4fc06d770 0x7fb4fc0b2250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00c7c0 tx=0x7fb4ec00cc80 comp rx=0 tx=0).stop 2024-04-22T07:04:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.774+0000 7fb500c5c640 1 -- 172.21.15.92:0/3061245997 shutdown_connections 2024-04-22T07:04:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.774+0000 7fb500c5c640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb4dc06c8f0 0x7fb4dc06edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.774+0000 7fb500c5c640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb4fc158390 0x7fb4fc0afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.774+0000 7fb500c5c640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb4fc0c9730 0x7fb4fc1f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.774+0000 7fb500c5c640 1 --2- 172.21.15.92:0/3061245997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4fc06d770 0x7fb4fc0b2250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.774+0000 7fb500c5c640 1 -- 172.21.15.92:0/3061245997 >> 172.21.15.92:0/3061245997 conn(0x7fb4fc0676b0 msgr2=0x7fb4fc06f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.774+0000 7fb500c5c640 1 -- 172.21.15.92:0/3061245997 shutdown_connections 2024-04-22T07:04:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.774+0000 7fb500c5c640 1 -- 172.21.15.92:0/3061245997 wait complete. 2024-04-22T07:04:02.799 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-22T07:04:02.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.905+0000 7fb779bd5640 1 Processor -- start 2024-04-22T07:04:02.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.905+0000 7fb779bd5640 1 -- start start 2024-04-22T07:04:02.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb779bd5640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7740d0b70 0x7fb774154a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:02.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb779bd5640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb77415e9b0 0x7fb774154f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:02.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb779bd5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77415ab30 con 0x7fb7740d0f50 2024-04-22T07:04:02.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb779bd5640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb77415ba00 con 0x7fb77415e9b0 2024-04-22T07:04:02.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb779bd5640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb77415adf0 con 0x7fb7740d0b70 2024-04-22T07:04:02.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb772ffd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb7740d0f50 0x7fb77415e2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39042/0 (socket says 172.21.15.92:39042) 2024-04-22T07:04:02.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb772ffd640 1 -- 172.21.15.92:0/267626900 learned_addr learned my addr 172.21.15.92:0/267626900 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb7737fe640 1 --2- 172.21.15.92:0/267626900 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7740d0b70 0x7fb774154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb773fff640 1 --2- 172.21.15.92:0/267626900 >> v2:172.21.15.145:3300/0 conn(0x7fb77415e9b0 0x7fb774154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb7727fc640 1 -- 172.21.15.92:0/267626900 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367366210 0 0) 0x7fb77415ab30 con 0x7fb7740d0f50 2024-04-22T07:04:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb7727fc640 1 -- 172.21.15.92:0/267626900 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb75c003520 con 0x7fb7740d0f50 2024-04-22T07:04:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb7737fe640 1 -- 172.21.15.92:0/267626900 >> v2:172.21.15.145:3300/0 conn(0x7fb77415e9b0 msgr2=0x7fb774154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb7737fe640 1 --2- 172.21.15.92:0/267626900 >> v2:172.21.15.145:3300/0 conn(0x7fb77415e9b0 0x7fb774154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb7737fe640 1 -- 172.21.15.92:0/267626900 >> v1:172.21.15.92:6789/0 conn(0x7fb7740d0f50 legacy=0x7fb77415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:02.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb7727fc640 1 -- 172.21.15.92:0/267626900 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2736117977 0 0) 0x7fb75c003520 con 0x7fb7740d0f50 2024-04-22T07:04:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.906+0000 7fb7737fe640 1 -- 172.21.15.92:0/267626900 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77415b0d0 con 0x7fb7740d0b70 2024-04-22T07:04:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.907+0000 7fb7737fe640 1 --2- 172.21.15.92:0/267626900 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7740d0b70 0x7fb774154a00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb7640097a0 tx=0x7fb76402b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c5ff3f20d921b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.907+0000 7fb7727fc640 1 -- 172.21.15.92:0/267626900 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764037070 con 0x7fb7740d0b70 2024-04-22T07:04:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.907+0000 7fb7727fc640 1 -- 172.21.15.92:0/267626900 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb764030510 con 0x7fb7740d0b70 2024-04-22T07:04:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.907+0000 7fb7727fc640 1 -- 172.21.15.92:0/267626900 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768001670 con 0x7fb7740d0b70 2024-04-22T07:04:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.908+0000 7fb779bd5640 1 -- 172.21.15.92:0/267626900 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7740d0b70 msgr2=0x7fb774154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.908+0000 7fb779bd5640 1 --2- 172.21.15.92:0/267626900 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7740d0b70 0x7fb774154a00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb7640097a0 tx=0x7fb76402b710 comp rx=0 tx=0).stop 2024-04-22T07:04:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.908+0000 7fb779bd5640 1 -- 172.21.15.92:0/267626900 shutdown_connections 2024-04-22T07:04:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.908+0000 7fb779bd5640 1 --2- 172.21.15.92:0/267626900 >> v2:172.21.15.145:3300/0 conn(0x7fb77415e9b0 0x7fb774154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:02.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.908+0000 7fb779bd5640 1 --2- 172.21.15.92:0/267626900 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7740d0b70 0x7fb774154a00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:02.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.908+0000 7fb779bd5640 1 -- 172.21.15.92:0/267626900 >> 172.21.15.92:0/267626900 conn(0x7fb7740736b0 msgr2=0x7fb774073ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:02.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.908+0000 7fb779bd5640 1 -- 172.21.15.92:0/267626900 shutdown_connections 2024-04-22T07:04:02.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.908+0000 7fb779bd5640 1 -- 172.21.15.92:0/267626900 wait complete. 2024-04-22T07:04:02.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.909+0000 7fb779bd5640 1 Processor -- start 2024-04-22T07:04:02.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.909+0000 7fb779bd5640 1 -- start start 2024-04-22T07:04:02.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb779bd5640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7740d0b70 0x7fb77415d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:02.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb779bd5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7740d0f50 0x7fb7741f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:02.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb779bd5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb77415e9b0 0x7fb7741f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:02.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb779bd5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb7740cae00 con 0x7fb7740d0f50 2024-04-22T07:04:02.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb779bd5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb77415b690 con 0x7fb77415e9b0 2024-04-22T07:04:02.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb779bd5640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb7740caf80 con 0x7fb7740d0b70 2024-04-22T07:04:02.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb772ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7740d0f50 0x7fb7741f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:02.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb772ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7740d0f50 0x7fb7741f3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45014/0 (socket says 172.21.15.92:45014) 2024-04-22T07:04:02.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb772ffd640 1 -- 172.21.15.92:0/1007010933 learned_addr learned my addr 172.21.15.92:0/1007010933 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:02.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb773fff640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb77415e9b0 0x7fb7741f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:02.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb7737fe640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7740d0b70 0x7fb77415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:02.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb772ffd640 1 -- 172.21.15.92:0/1007010933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7740d0b70 msgr2=0x7fb77415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:02.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb772ffd640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7740d0b70 0x7fb77415d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:02.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb772ffd640 1 -- 172.21.15.92:0/1007010933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb77415e9b0 msgr2=0x7fb7741f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:02.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb772ffd640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb77415e9b0 0x7fb7741f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:02.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb772ffd640 1 -- 172.21.15.92:0/1007010933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7740c7990 con 0x7fb7740d0f50 2024-04-22T07:04:02.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb772ffd640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7740d0f50 0x7fb7741f3c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb76800c760 tx=0x7fb76800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:02.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb75bfff640 1 -- 172.21.15.92:0/1007010933 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768017070 con 0x7fb7740d0f50 2024-04-22T07:04:02.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb75bfff640 1 -- 172.21.15.92:0/1007010933 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb768005ad0 con 0x7fb7740d0f50 2024-04-22T07:04:02.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb779bd5640 1 -- 172.21.15.92:0/1007010933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7740c9390 con 0x7fb7740d0f50 2024-04-22T07:04:02.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.910+0000 7fb75bfff640 1 -- 172.21.15.92:0/1007010933 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768007070 con 0x7fb7740d0f50 2024-04-22T07:04:02.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.911+0000 7fb779bd5640 1 -- 172.21.15.92:0/1007010933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7740c9950 con 0x7fb7740d0f50 2024-04-22T07:04:02.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.911+0000 7fb779bd5640 1 -- 172.21.15.92:0/1007010933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb774069930 con 0x7fb7740d0f50 2024-04-22T07:04:02.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.912+0000 7fb75bfff640 1 -- 172.21.15.92:0/1007010933 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb76801a070 con 0x7fb7740d0f50 2024-04-22T07:04:02.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.912+0000 7fb75bfff640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb76006c920 0x7fb76006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:02.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.912+0000 7fb7737fe640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb76006c920 0x7fb76006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:02.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.912+0000 7fb75bfff640 1 -- 172.21.15.92:0/1007010933 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fb76808b060 con 0x7fb7740d0f50 2024-04-22T07:04:02.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.913+0000 7fb7737fe640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb76006c920 0x7fb76006ede0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb764034000 tx=0x7fb764008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:02.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:02.917+0000 7fb75bfff640 1 -- 172.21.15.92:0/1007010933 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb768059460 con 0x7fb7740d0f50 2024-04-22T07:04:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.127+0000 7fb779bd5640 1 -- 172.21.15.92:0/1007010933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb7740c9b20 con 0x7fb7740d0f50 2024-04-22T07:04:03.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.773+0000 7fb75bfff640 1 -- 172.21.15.92:0/1007010933 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fb768058d60 con 0x7fb7740d0f50 2024-04-22T07:04:03.771 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 2 and data pool 3 2024-04-22T07:04:03.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.775+0000 7fb779bd5640 1 -- 172.21.15.92:0/1007010933 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb76006c920 msgr2=0x7fb76006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:03.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.775+0000 7fb779bd5640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb76006c920 0x7fb76006ede0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb764034000 tx=0x7fb764008030 comp rx=0 tx=0).stop 2024-04-22T07:04:03.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.775+0000 7fb779bd5640 1 -- 172.21.15.92:0/1007010933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7740d0f50 msgr2=0x7fb7741f3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:03.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.775+0000 7fb779bd5640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7740d0f50 0x7fb7741f3c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb76800c760 tx=0x7fb76800cc20 comp rx=0 tx=0).stop 2024-04-22T07:04:03.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.775+0000 7fb779bd5640 1 -- 172.21.15.92:0/1007010933 shutdown_connections 2024-04-22T07:04:03.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.775+0000 7fb779bd5640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb76006c920 0x7fb76006ede0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:03.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.775+0000 7fb779bd5640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb77415e9b0 0x7fb7741f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:03.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.775+0000 7fb779bd5640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7740d0f50 0x7fb7741f3c90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:03.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.775+0000 7fb779bd5640 1 --2- 172.21.15.92:0/1007010933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7740d0b70 0x7fb77415d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:03.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.775+0000 7fb779bd5640 1 -- 172.21.15.92:0/1007010933 >> 172.21.15.92:0/1007010933 conn(0x7fb7740736b0 msgr2=0x7fb774152ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:03.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.776+0000 7fb779bd5640 1 -- 172.21.15.92:0/1007010933 shutdown_connections 2024-04-22T07:04:03.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.776+0000 7fb779bd5640 1 -- 172.21.15.92:0/1007010933 wait complete. 2024-04-22T07:04:03.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:03.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.898+0000 7f4edb04e640 1 Processor -- start 2024-04-22T07:04:03.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.898+0000 7f4edb04e640 1 -- start start 2024-04-22T07:04:03.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4edb04e640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4ed40ce2f0 0x7f4ed406f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:03.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4edb04e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ed40c9db0 0x7f4ed40ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:03.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4edb04e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ed415ad90 0x7f4ed415b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:03.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4edb04e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed40c9a10 con 0x7f4ed415ad90 2024-04-22T07:04:03.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4edb04e640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed415b760 con 0x7f4ed40ce2f0 2024-04-22T07:04:03.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4edb04e640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed40c8e00 con 0x7f4ed40c9db0 2024-04-22T07:04:03.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4ed95eb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ed415ad90 0x7f4ed415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:03.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4ed95eb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ed415ad90 0x7f4ed415b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45018/0 (socket says 172.21.15.92:45018) 2024-04-22T07:04:03.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4ed95eb640 1 -- 172.21.15.92:0/1878663220 learned_addr learned my addr 172.21.15.92:0/1878663220 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:03.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4ed8dea640 1 --2- 172.21.15.92:0/1878663220 >> v2:172.21.15.145:3300/0 conn(0x7f4ed40ce2f0 0x7f4ed406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:03.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4ed3fff640 1 --2- 172.21.15.92:0/1878663220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ed40c9db0 0x7f4ed40ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:03.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4ed95eb640 1 -- 172.21.15.92:0/1878663220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ed40c9db0 msgr2=0x7f4ed40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:03.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4ed95eb640 1 --2- 172.21.15.92:0/1878663220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ed40c9db0 0x7f4ed40ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:03.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4ed95eb640 1 -- 172.21.15.92:0/1878663220 >> v2:172.21.15.145:3300/0 conn(0x7f4ed40ce2f0 msgr2=0x7f4ed406f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:03.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4ed95eb640 1 --2- 172.21.15.92:0/1878663220 >> v2:172.21.15.145:3300/0 conn(0x7f4ed40ce2f0 0x7f4ed406f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:03.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.899+0000 7f4ed95eb640 1 -- 172.21.15.92:0/1878663220 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed40c90e0 con 0x7f4ed415ad90 2024-04-22T07:04:03.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.900+0000 7f4ed95eb640 1 --2- 172.21.15.92:0/1878663220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed415ad90 0x7f4ed415b220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4072ef0 tx=0x7f4ecc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31944a2fb6d47662 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:03.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.900+0000 7f4ed37fe640 1 -- 172.21.15.92:0/1878663220 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc02cb00 con 0x7f4ed415ad90 2024-04-22T07:04:03.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.900+0000 7f4ed37fe640 1 -- 172.21.15.92:0/1878663220 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ecc02cca0 con 0x7f4ed415ad90 2024-04-22T07:04:03.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.900+0000 7f4ed37fe640 1 -- 172.21.15.92:0/1878663220 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc034610 con 0x7f4ed415ad90 2024-04-22T07:04:03.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.901+0000 7f4edb04e640 1 -- 172.21.15.92:0/1878663220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed415ad90 msgr2=0x7f4ed415b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:03.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.901+0000 7f4edb04e640 1 --2- 172.21.15.92:0/1878663220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed415ad90 0x7f4ed415b220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4072ef0 tx=0x7f4ecc02bf60 comp rx=0 tx=0).stop 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.901+0000 7f4edb04e640 1 -- 172.21.15.92:0/1878663220 shutdown_connections 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.901+0000 7f4edb04e640 1 --2- 172.21.15.92:0/1878663220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed415ad90 0x7f4ed415b220 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.901+0000 7f4edb04e640 1 --2- 172.21.15.92:0/1878663220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ed40c9db0 0x7f4ed40ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.901+0000 7f4edb04e640 1 --2- 172.21.15.92:0/1878663220 >> v2:172.21.15.145:3300/0 conn(0x7f4ed40ce2f0 0x7f4ed406f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.901+0000 7f4edb04e640 1 -- 172.21.15.92:0/1878663220 >> 172.21.15.92:0/1878663220 conn(0x7f4ed406db80 msgr2=0x7f4ed406df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.901+0000 7f4edb04e640 1 -- 172.21.15.92:0/1878663220 shutdown_connections 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.901+0000 7f4edb04e640 1 -- 172.21.15.92:0/1878663220 wait complete. 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.902+0000 7f4edb04e640 1 Processor -- start 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.902+0000 7f4edb04e640 1 -- start start 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.902+0000 7f4edb04e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40c9db0 0x7f4ed414e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4edb04e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ed40ce2f0 0x7f4ed414ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4edb04e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ed415ad90 0x7f4ed414fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:03.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4edb04e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed406d910 con 0x7f4ed40c9db0 2024-04-22T07:04:03.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4edb04e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed40c96a0 con 0x7f4ed415ad90 2024-04-22T07:04:03.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4edb04e640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed4070010 con 0x7f4ed40ce2f0 2024-04-22T07:04:03.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4ed8dea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40c9db0 0x7f4ed414e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:03.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4ed8dea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40c9db0 0x7f4ed414e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45026/0 (socket says 172.21.15.92:45026) 2024-04-22T07:04:03.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4ed8dea640 1 -- 172.21.15.92:0/665674323 learned_addr learned my addr 172.21.15.92:0/665674323 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:03.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4ed8dea640 1 -- 172.21.15.92:0/665674323 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ed40ce2f0 msgr2=0x7f4ed414ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:03.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4ed8dea640 1 --2- 172.21.15.92:0/665674323 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ed40ce2f0 0x7f4ed414ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:03.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4ed8dea640 1 -- 172.21.15.92:0/665674323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ed415ad90 msgr2=0x7f4ed414fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4ed8dea640 1 --2- 172.21.15.92:0/665674323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ed415ad90 0x7f4ed414fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4ed8dea640 1 -- 172.21.15.92:0/665674323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed41f52c0 con 0x7f4ed40c9db0 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4ed8dea640 1 --2- 172.21.15.92:0/665674323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40c9db0 0x7f4ed414e870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800d830 tx=0x7f4ec800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4ed17fa640 1 -- 172.21.15.92:0/665674323 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec800b6c0 con 0x7f4ed40c9db0 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4ed17fa640 1 -- 172.21.15.92:0/665674323 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec8019070 con 0x7f4ed40c9db0 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4edb04e640 1 -- 172.21.15.92:0/665674323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed41585d0 con 0x7f4ed40c9db0 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4ed17fa640 1 -- 172.21.15.92:0/665674323 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec800ba80 con 0x7f4ed40c9db0 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.903+0000 7f4edb04e640 1 -- 172.21.15.92:0/665674323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed41f47e0 con 0x7f4ed40c9db0 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.904+0000 7f4edb04e640 1 -- 172.21.15.92:0/665674323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed406bdc0 con 0x7f4ed40c9db0 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.905+0000 7f4ed17fa640 1 -- 172.21.15.92:0/665674323 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ec800bc20 con 0x7f4ed40c9db0 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.905+0000 7f4ed17fa640 1 --2- 172.21.15.92:0/665674323 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4eb806c920 0x7f4eb806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.905+0000 7f4ed3fff640 1 --2- 172.21.15.92:0/665674323 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4eb806c920 0x7f4eb806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.905+0000 7f4ed17fa640 1 -- 172.21.15.92:0/665674323 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f4ec808bf90 con 0x7f4ed40c9db0 2024-04-22T07:04:03.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.906+0000 7f4ed3fff640 1 --2- 172.21.15.92:0/665674323 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4eb806c920 0x7f4eb806ede0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ed406fd50 tx=0x7f4ec0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:03.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:03.910+0000 7f4ed17fa640 1 -- 172.21.15.92:0/665674323 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ed406bdc0 con 0x7f4ed40c9db0 2024-04-22T07:04:04.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.086+0000 7f4edb04e640 1 -- 172.21.15.92:0/665674323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ed406c040 con 0x7f4ed40c9db0 2024-04-22T07:04:04.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.087+0000 7f4ed17fa640 1 -- 172.21.15.92:0/665674323 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16570 (secure 0 0 0) 0x7f4ec8083e20 con 0x7f4ed40c9db0 2024-04-22T07:04:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:04.086 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":19,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:04:03.129129+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:04.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.089+0000 7f4edb04e640 1 -- 172.21.15.92:0/665674323 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4eb806c920 msgr2=0x7f4eb806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.089+0000 7f4edb04e640 1 --2- 172.21.15.92:0/665674323 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4eb806c920 0x7f4eb806ede0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ed406fd50 tx=0x7f4ec0008040 comp rx=0 tx=0).stop 2024-04-22T07:04:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.089+0000 7f4edb04e640 1 -- 172.21.15.92:0/665674323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40c9db0 msgr2=0x7f4ed414e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.089+0000 7f4edb04e640 1 --2- 172.21.15.92:0/665674323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40c9db0 0x7f4ed414e870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800d830 tx=0x7f4ec800dcf0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.089+0000 7f4edb04e640 1 -- 172.21.15.92:0/665674323 shutdown_connections 2024-04-22T07:04:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.089+0000 7f4edb04e640 1 --2- 172.21.15.92:0/665674323 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4eb806c920 0x7f4eb806ede0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.089+0000 7f4edb04e640 1 --2- 172.21.15.92:0/665674323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ed415ad90 0x7f4ed414fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.089+0000 7f4edb04e640 1 --2- 172.21.15.92:0/665674323 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ed40ce2f0 0x7f4ed414ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.089+0000 7f4edb04e640 1 --2- 172.21.15.92:0/665674323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ed40c9db0 0x7f4ed414e870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.089+0000 7f4edb04e640 1 -- 172.21.15.92:0/665674323 >> 172.21.15.92:0/665674323 conn(0x7f4ed406db80 msgr2=0x7f4ed415a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.089+0000 7f4edb04e640 1 -- 172.21.15.92:0/665674323 shutdown_connections 2024-04-22T07:04:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.090+0000 7f4edb04e640 1 -- 172.21.15.92:0/665674323 wait complete. 2024-04-22T07:04:04.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:04.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.196+0000 7fab50c9a640 1 Processor -- start 2024-04-22T07:04:04.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.196+0000 7fab50c9a640 1 -- start start 2024-04-22T07:04:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.197+0000 7fab50c9a640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fab4c1594f0 0x7fab4c06fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.197+0000 7fab50c9a640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab4c0c72b0 con 0x7fab4c152d80 2024-04-22T07:04:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.197+0000 7fab50c9a640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab4c153160 con 0x7fab4c0c87e0 2024-04-22T07:04:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.197+0000 7fab50c9a640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fab4c06ce60 con 0x7fab4c1594f0 2024-04-22T07:04:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.197+0000 7fab4a59c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fab4c152d80 0x7fab4c0ca020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39050/0 (socket says 172.21.15.92:39050) 2024-04-22T07:04:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.197+0000 7fab4a59c640 1 -- 172.21.15.92:0/3971604253 learned_addr learned my addr 172.21.15.92:0/3971604253 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.197+0000 7fab4ad9d640 1 --2- 172.21.15.92:0/3971604253 >> v2:172.21.15.145:3300/0 conn(0x7fab4c1594f0 0x7fab4c06fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.197+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619729050 0 0) 0x7fab4c0c72b0 con 0x7fab4c152d80 2024-04-22T07:04:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.197+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab34003520 con 0x7fab4c152d80 2024-04-22T07:04:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.197+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3227590213 0 0) 0x7fab34003520 con 0x7fab4c152d80 2024-04-22T07:04:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.197+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 >> v2:172.21.15.145:3300/0 conn(0x7fab4c1594f0 msgr2=0x7fab4c06fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:04.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4959a640 1 --2- 172.21.15.92:0/3971604253 >> v2:172.21.15.145:3300/0 conn(0x7fab4c1594f0 0x7fab4c06fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 >> v1:172.21.15.145:6789/0 conn(0x7fab4c0c87e0 legacy=0x7fab4c158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c0c6370 con 0x7fab4c152d80 2024-04-22T07:04:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7fab3c0019f0 con 0x7fab4c152d80 2024-04-22T07:04:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 >> v1:172.21.15.92:6789/0 conn(0x7fab4c152d80 legacy=0x7fab4c0ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4959a640 1 --2- 172.21.15.92:0/3971604253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab340039b0 0x7fab34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4959a640 1 --2- 172.21.15.92:0/3971604253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab34004800 0x7fab34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4959a640 1 --2- 172.21.15.92:0/3971604253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab340051c0 0x7fab3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c06ce60 con 0x7fab340039b0 2024-04-22T07:04:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fab3400c070 con 0x7fab34004800 2024-04-22T07:04:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fab3400c270 con 0x7fab340051c0 2024-04-22T07:04:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4a59c640 1 --2- 172.21.15.92:0/3971604253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab340039b0 0x7fab34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab49d9b640 1 --2- 172.21.15.92:0/3971604253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab34004800 0x7fab34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4ad9d640 1 --2- 172.21.15.92:0/3971604253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab340051c0 0x7fab3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4a59c640 1 -- 172.21.15.92:0/3971604253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab340051c0 msgr2=0x7fab3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4a59c640 1 --2- 172.21.15.92:0/3971604253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab340051c0 0x7fab3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4a59c640 1 -- 172.21.15.92:0/3971604253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab34004800 msgr2=0x7fab34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4a59c640 1 --2- 172.21.15.92:0/3971604253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab34004800 0x7fab34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4a59c640 1 -- 172.21.15.92:0/3971604253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab34003520 con 0x7fab340039b0 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.198+0000 7fab4ad9d640 1 -- 172.21.15.92:0/3971604253 reap_dead start 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.199+0000 7fab4a59c640 1 --2- 172.21.15.92:0/3971604253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab340039b0 0x7fab34003dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00a860 tx=0x7fab34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8077def0f9bdd256 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.199+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00a480 con 0x7fab340039b0 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.199+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab3c034630 con 0x7fab340039b0 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.199+0000 7fab4959a640 1 -- 172.21.15.92:0/3971604253 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c034aa0 con 0x7fab340039b0 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.199+0000 7fab50c9a640 1 -- 172.21.15.92:0/3971604253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab340039b0 msgr2=0x7fab34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.199+0000 7fab50c9a640 1 --2- 172.21.15.92:0/3971604253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab340039b0 0x7fab34003dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00a860 tx=0x7fab34005830 comp rx=0 tx=0).stop 2024-04-22T07:04:04.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.199+0000 7fab50c9a640 1 -- 172.21.15.92:0/3971604253 shutdown_connections 2024-04-22T07:04:04.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.199+0000 7fab50c9a640 1 --2- 172.21.15.92:0/3971604253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab340039b0 0x7fab34003dd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.199+0000 7fab50c9a640 1 -- 172.21.15.92:0/3971604253 >> 172.21.15.92:0/3971604253 conn(0x7fab4c0cc220 msgr2=0x7fab4c0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:04.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.199+0000 7fab50c9a640 1 -- 172.21.15.92:0/3971604253 shutdown_connections 2024-04-22T07:04:04.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.199+0000 7fab50c9a640 1 -- 172.21.15.92:0/3971604253 wait complete. 2024-04-22T07:04:04.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.201+0000 7fab50c9a640 1 Processor -- start 2024-04-22T07:04:04.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.201+0000 7fab50c9a640 1 -- start start 2024-04-22T07:04:04.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.201+0000 7fab50c9a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab340039b0 0x7fab4c0733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.201+0000 7fab50c9a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4c1578a0 0x7fab4c145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.201+0000 7fab50c9a640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab4c140540 0x7fab4c146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.201+0000 7fab50c9a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c0c5db0 con 0x7fab4c1578a0 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.201+0000 7fab50c9a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c0c5ad0 con 0x7fab340039b0 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.201+0000 7fab50c9a640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fab4c0c6090 con 0x7fab4c140540 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab49d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4c1578a0 0x7fab4c145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab49d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4c1578a0 0x7fab4c145f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45046/0 (socket says 172.21.15.92:45046) 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab49d9b640 1 -- 172.21.15.92:0/4096476922 learned_addr learned my addr 172.21.15.92:0/4096476922 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab4ad9d640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab4c140540 0x7fab4c146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab4a59c640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab340039b0 0x7fab4c0733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab49d9b640 1 -- 172.21.15.92:0/4096476922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab4c140540 msgr2=0x7fab4c146470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab49d9b640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab4c140540 0x7fab4c146470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab49d9b640 1 -- 172.21.15.92:0/4096476922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab340039b0 msgr2=0x7fab4c0733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab49d9b640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab340039b0 0x7fab4c0733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab49d9b640 1 -- 172.21.15.92:0/4096476922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c157f40 con 0x7fab4c1578a0 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab4a59c640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab340039b0 0x7fab4c0733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab49d9b640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4c1578a0 0x7fab4c145f30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab4000f650 tx=0x7fab4000fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab3affd640 1 -- 172.21.15.92:0/4096476922 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab400052e0 con 0x7fab4c1578a0 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab3affd640 1 -- 172.21.15.92:0/4096476922 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab40018070 con 0x7fab4c1578a0 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab3affd640 1 -- 172.21.15.92:0/4096476922 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40003810 con 0x7fab4c1578a0 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab50c9a640 1 -- 172.21.15.92:0/4096476922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4c0729d0 con 0x7fab4c1578a0 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.202+0000 7fab50c9a640 1 -- 172.21.15.92:0/4096476922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4c156b40 con 0x7fab4c1578a0 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.203+0000 7fab50c9a640 1 -- 172.21.15.92:0/4096476922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab4c152930 con 0x7fab4c1578a0 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.203+0000 7fab3affd640 1 -- 172.21.15.92:0/4096476922 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab400039b0 con 0x7fab4c1578a0 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.204+0000 7fab3affd640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fab2c06c8f0 0x7fab2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.204+0000 7fab3affd640 1 -- 172.21.15.92:0/4096476922 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fab4008b790 con 0x7fab4c1578a0 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.204+0000 7fab4a59c640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fab2c06c8f0 0x7fab2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.204+0000 7fab4a59c640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fab2c06c8f0 0x7fab2c06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab3c034000 tx=0x7fab3c036040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:04.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.208+0000 7fab3affd640 1 -- 172.21.15.92:0/4096476922 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab4c152930 con 0x7fab4c1578a0 2024-04-22T07:04:04.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.384+0000 7fab50c9a640 1 -- 172.21.15.92:0/4096476922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab4c149870 con 0x7fab4c1578a0 2024-04-22T07:04:04.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.385+0000 7fab3affd640 1 -- 172.21.15.92:0/4096476922 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16570 (secure 0 0 0) 0x7fab4c149870 con 0x7fab4c1578a0 2024-04-22T07:04:04.383 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:04.384 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":19,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:04:03.129129+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.387+0000 7fab50c9a640 1 -- 172.21.15.92:0/4096476922 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fab2c06c8f0 msgr2=0x7fab2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.387+0000 7fab50c9a640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fab2c06c8f0 0x7fab2c06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab3c034000 tx=0x7fab3c036040 comp rx=0 tx=0).stop 2024-04-22T07:04:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.387+0000 7fab50c9a640 1 -- 172.21.15.92:0/4096476922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4c1578a0 msgr2=0x7fab4c145f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.387+0000 7fab50c9a640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4c1578a0 0x7fab4c145f30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab4000f650 tx=0x7fab4000fa50 comp rx=0 tx=0).stop 2024-04-22T07:04:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.387+0000 7fab50c9a640 1 -- 172.21.15.92:0/4096476922 shutdown_connections 2024-04-22T07:04:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.387+0000 7fab50c9a640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fab2c06c8f0 0x7fab2c06edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.387+0000 7fab50c9a640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab4c140540 0x7fab4c146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.387+0000 7fab50c9a640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4c1578a0 0x7fab4c145f30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.387+0000 7fab50c9a640 1 --2- 172.21.15.92:0/4096476922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab340039b0 0x7fab4c0733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.387+0000 7fab50c9a640 1 -- 172.21.15.92:0/4096476922 >> 172.21.15.92:0/4096476922 conn(0x7fab4c0cc220 msgr2=0x7fab4c06f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.387+0000 7fab50c9a640 1 -- 172.21.15.92:0/4096476922 shutdown_connections 2024-04-22T07:04:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.387+0000 7fab50c9a640 1 -- 172.21.15.92:0/4096476922 wait complete. 2024-04-22T07:04:04.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-22T07:04:04.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.497+0000 7f554c185640 1 Processor -- start 2024-04-22T07:04:04.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.497+0000 7f554c185640 1 -- start start 2024-04-22T07:04:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.497+0000 7f554c185640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5544152ed0 0x7f55440b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f554c185640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55440caa80 con 0x7f55440c5270 2024-04-22T07:04:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f554c185640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55440ce580 con 0x7f55440b09e0 2024-04-22T07:04:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f554c185640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f55440711c0 con 0x7f5544152ed0 2024-04-22T07:04:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5549720640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f55440c5270 0x7f55440ca370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39056/0 (socket says 172.21.15.92:39056) 2024-04-22T07:04:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5549720640 1 -- 172.21.15.92:0/1742457494 learned_addr learned my addr 172.21.15.92:0/1742457494 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1876340775 0 0) 0x7f55440caa80 con 0x7f55440c5270 2024-04-22T07:04:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5549f21640 1 --2- 172.21.15.92:0/1742457494 >> v2:172.21.15.145:3300/0 conn(0x7f5544152ed0 0x7f55440b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5530003520 con 0x7f55440c5270 2024-04-22T07:04:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4010247529 0 0) 0x7f5530003520 con 0x7f55440c5270 2024-04-22T07:04:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 >> v2:172.21.15.145:3300/0 conn(0x7f5544152ed0 msgr2=0x7f55440b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 --2- 172.21.15.92:0/1742457494 >> v2:172.21.15.145:3300/0 conn(0x7f5544152ed0 0x7f55440b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 >> v1:172.21.15.145:6789/0 conn(0x7f55440b09e0 legacy=0x7f5544070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55441532b0 con 0x7f55440c5270 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f5534001670 con 0x7f55440c5270 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5549f21640 1 --2- 172.21.15.92:0/1742457494 >> v2:172.21.15.145:3300/0 conn(0x7f5544152ed0 0x7f55440b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 >> v1:172.21.15.92:6789/0 conn(0x7f55440c5270 legacy=0x7f55440ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 --2- 172.21.15.92:0/1742457494 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55300039b0 0x7f5530003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 --2- 172.21.15.92:0/1742457494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5530004800 0x7f5530004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 --2- 172.21.15.92:0/1742457494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 0x7f553000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f55440711c0 con 0x7f55300052e0 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f553000c1b0 con 0x7f5530004800 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f553000c3b0 con 0x7f55300039b0 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f554a722640 1 --2- 172.21.15.92:0/1742457494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 0x7f553000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5549720640 1 --2- 172.21.15.92:0/1742457494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5530004800 0x7f5530004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.498+0000 7f5549f21640 1 --2- 172.21.15.92:0/1742457494 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55300039b0 0x7f5530003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.499+0000 7f554a722640 1 -- 172.21.15.92:0/1742457494 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55300039b0 msgr2=0x7f5530003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.499+0000 7f554a722640 1 --2- 172.21.15.92:0/1742457494 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55300039b0 0x7f5530003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.499+0000 7f554a722640 1 -- 172.21.15.92:0/1742457494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5530004800 msgr2=0x7f5530004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.499+0000 7f554a722640 1 --2- 172.21.15.92:0/1742457494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5530004800 0x7f5530004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.499+0000 7f554a722640 1 -- 172.21.15.92:0/1742457494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5530003520 con 0x7f55300052e0 2024-04-22T07:04:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.499+0000 7f554a722640 1 -- 172.21.15.92:0/1742457494 reap_dead start 2024-04-22T07:04:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.499+0000 7f554a722640 1 --2- 172.21.15.92:0/1742457494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 0x7f553000b9c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f553c001e20 tx=0x7f5530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70011de7f2a21832 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.499+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c001a70 con 0x7f55300052e0 2024-04-22T07:04:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.499+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c003b70 con 0x7f55300052e0 2024-04-22T07:04:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.499+0000 7f5548f1f640 1 -- 172.21.15.92:0/1742457494 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c0050d0 con 0x7f55300052e0 2024-04-22T07:04:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.500+0000 7f554c185640 1 -- 172.21.15.92:0/1742457494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 msgr2=0x7f553000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.500+0000 7f554c185640 1 --2- 172.21.15.92:0/1742457494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 0x7f553000b9c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f553c001e20 tx=0x7f5530005830 comp rx=0 tx=0).stop 2024-04-22T07:04:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.500+0000 7f554c185640 1 -- 172.21.15.92:0/1742457494 shutdown_connections 2024-04-22T07:04:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.500+0000 7f554c185640 1 --2- 172.21.15.92:0/1742457494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 0x7f553000b9c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.500+0000 7f554c185640 1 -- 172.21.15.92:0/1742457494 >> 172.21.15.92:0/1742457494 conn(0x7f554406db80 msgr2=0x7f554406df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:04.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.500+0000 7f554c185640 1 -- 172.21.15.92:0/1742457494 shutdown_connections 2024-04-22T07:04:04.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.500+0000 7f554c185640 1 -- 172.21.15.92:0/1742457494 wait complete. 2024-04-22T07:04:04.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.501+0000 7f554c185640 1 Processor -- start 2024-04-22T07:04:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.501+0000 7f554c185640 1 -- start start 2024-04-22T07:04:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f554c185640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 0x7f554414e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f554c185640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5544152520 0x7f55441492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f554c185640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55440719a0 0x7f5544071e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f554c185640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f55440c39a0 con 0x7f55300052e0 2024-04-22T07:04:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f554c185640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f55440c5760 con 0x7f5544152520 2024-04-22T07:04:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f554c185640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f55440ce820 con 0x7f55440719a0 2024-04-22T07:04:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f5549f21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 0x7f554414e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f5549f21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 0x7f554414e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45076/0 (socket says 172.21.15.92:45076) 2024-04-22T07:04:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f5549f21640 1 -- 172.21.15.92:0/1355895940 learned_addr learned my addr 172.21.15.92:0/1355895940 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f554a722640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55440719a0 0x7f5544071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f5549720640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5544152520 0x7f55441492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f5549f21640 1 -- 172.21.15.92:0/1355895940 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55440719a0 msgr2=0x7f5544071e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f5549f21640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55440719a0 0x7f5544071e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f5549f21640 1 -- 172.21.15.92:0/1355895940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5544152520 msgr2=0x7f55441492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f5549f21640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5544152520 0x7f55441492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f5549f21640 1 -- 172.21.15.92:0/1355895940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554414d180 con 0x7f55300052e0 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.502+0000 7f5549f21640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 0x7f554414e770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55400115c0 tx=0x7f5540011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.503+0000 7f553a7fc640 1 -- 172.21.15.92:0/1355895940 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540019070 con 0x7f55300052e0 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.503+0000 7f553a7fc640 1 -- 172.21.15.92:0/1355895940 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5540004220 con 0x7f55300052e0 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.503+0000 7f553a7fc640 1 -- 172.21.15.92:0/1355895940 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554000f040 con 0x7f55300052e0 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.503+0000 7f554c185640 1 -- 172.21.15.92:0/1355895940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55440b2e30 con 0x7f55300052e0 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.503+0000 7f554c185640 1 -- 172.21.15.92:0/1355895940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554414ed50 con 0x7f55300052e0 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.504+0000 7f554c185640 1 -- 172.21.15.92:0/1355895940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5544069930 con 0x7f55300052e0 2024-04-22T07:04:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.504+0000 7f553a7fc640 1 -- 172.21.15.92:0/1355895940 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f554000f6f0 con 0x7f55300052e0 2024-04-22T07:04:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.504+0000 7f553a7fc640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f552806c8f0 0x7f552806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.505+0000 7f5549720640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f552806c8f0 0x7f552806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.505+0000 7f553a7fc640 1 -- 172.21.15.92:0/1355895940 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f554008bde0 con 0x7f55300052e0 2024-04-22T07:04:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.505+0000 7f5549720640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f552806c8f0 0x7f552806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5534001050 tx=0x7f5534009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:04.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.509+0000 7f553a7fc640 1 -- 172.21.15.92:0/1355895940 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f554005a210 con 0x7f55300052e0 2024-04-22T07:04:04.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.720+0000 7f554c185640 1 -- 172.21.15.92:0/1355895940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5544070c70 con 0x7f55300052e0 2024-04-22T07:04:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.790+0000 7f553a7fc640 1 -- 172.21.15.92:0/1355895940 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7f5540059b10 con 0x7f55300052e0 2024-04-22T07:04:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.792+0000 7f554c185640 1 -- 172.21.15.92:0/1355895940 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f552806c8f0 msgr2=0x7f552806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.792+0000 7f554c185640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f552806c8f0 0x7f552806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5534001050 tx=0x7f5534009040 comp rx=0 tx=0).stop 2024-04-22T07:04:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.792+0000 7f554c185640 1 -- 172.21.15.92:0/1355895940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 msgr2=0x7f554414e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.792+0000 7f554c185640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 0x7f554414e770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55400115c0 tx=0x7f5540011a80 comp rx=0 tx=0).stop 2024-04-22T07:04:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.792+0000 7f554c185640 1 -- 172.21.15.92:0/1355895940 shutdown_connections 2024-04-22T07:04:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.792+0000 7f554c185640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f552806c8f0 0x7f552806edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.792+0000 7f554c185640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55440719a0 0x7f5544071e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.792+0000 7f554c185640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5544152520 0x7f55441492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.792+0000 7f554c185640 1 --2- 172.21.15.92:0/1355895940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55300052e0 0x7f554414e770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.792+0000 7f554c185640 1 -- 172.21.15.92:0/1355895940 >> 172.21.15.92:0/1355895940 conn(0x7f554406db80 msgr2=0x7f5544156f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.792+0000 7f554c185640 1 -- 172.21.15.92:0/1355895940 shutdown_connections 2024-04-22T07:04:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.792+0000 7f554c185640 1 -- 172.21.15.92:0/1355895940 wait complete. 2024-04-22T07:04:04.800 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300} 2024-04-22T07:04:04.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-22T07:04:04.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.904+0000 7f53a08fd640 1 Processor -- start 2024-04-22T07:04:04.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.904+0000 7f53a08fd640 1 -- start start 2024-04-22T07:04:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f53a08fd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f539c0d0f50 0x7f539c15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f53a08fd640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539c15ab30 con 0x7f539c0cb120 2024-04-22T07:04:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f53a08fd640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539c15adf0 con 0x7f539c0d0b70 2024-04-22T07:04:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f53a08fd640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f539c15ba00 con 0x7f539c0d0f50 2024-04-22T07:04:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f539ad9d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f539c0cb120 0x7f539c165430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39072/0 (socket says 172.21.15.92:39072) 2024-04-22T07:04:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f539ad9d640 1 -- 172.21.15.92:0/1818897216 learned_addr learned my addr 172.21.15.92:0/1818897216 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f5393fff640 1 --2- 172.21.15.92:0/1818897216 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f539c0d0f50 0x7f539c15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f5399d9b640 1 -- 172.21.15.92:0/1818897216 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3634240269 0 0) 0x7f539c15ab30 con 0x7f539c0cb120 2024-04-22T07:04:04.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f5399d9b640 1 -- 172.21.15.92:0/1818897216 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5388003520 con 0x7f539c0cb120 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f5399d9b640 1 -- 172.21.15.92:0/1818897216 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3752614934 0 0) 0x7f539c15adf0 con 0x7f539c0d0b70 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f5399d9b640 1 -- 172.21.15.92:0/1818897216 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f539c15ab30 con 0x7f539c0d0b70 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f5399d9b640 1 -- 172.21.15.92:0/1818897216 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1007166651 0 0) 0x7f5388003520 con 0x7f539c0cb120 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f5393fff640 1 -- 172.21.15.92:0/1818897216 >> v1:172.21.15.145:6789/0 conn(0x7f539c0d0b70 legacy=0x7f539c154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f5393fff640 1 -- 172.21.15.92:0/1818897216 >> v1:172.21.15.92:6789/0 conn(0x7f539c0cb120 legacy=0x7f539c165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.905+0000 7f5393fff640 1 -- 172.21.15.92:0/1818897216 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c15b0d0 con 0x7f539c0d0f50 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.906+0000 7f5393fff640 1 --2- 172.21.15.92:0/1818897216 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f539c0d0f50 0x7f539c15e2a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f53840097d0 tx=0x7f538402b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2f757446d2a5e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.906+0000 7f5399d9b640 1 -- 172.21.15.92:0/1818897216 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384037070 con 0x7f539c0d0f50 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.906+0000 7f5399d9b640 1 -- 172.21.15.92:0/1818897216 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5384030510 con 0x7f539c0d0f50 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.906+0000 7f5399d9b640 1 -- 172.21.15.92:0/1818897216 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53940019f0 con 0x7f539c0d0f50 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.907+0000 7f53a08fd640 1 -- 172.21.15.92:0/1818897216 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f539c0d0f50 msgr2=0x7f539c15e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.907+0000 7f53a08fd640 1 --2- 172.21.15.92:0/1818897216 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f539c0d0f50 0x7f539c15e2a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f53840097d0 tx=0x7f538402b770 comp rx=0 tx=0).stop 2024-04-22T07:04:04.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.907+0000 7f53a08fd640 1 -- 172.21.15.92:0/1818897216 shutdown_connections 2024-04-22T07:04:04.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.907+0000 7f53a08fd640 1 --2- 172.21.15.92:0/1818897216 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f539c0d0f50 0x7f539c15e2a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.907+0000 7f53a08fd640 1 -- 172.21.15.92:0/1818897216 >> 172.21.15.92:0/1818897216 conn(0x7f539c0736b0 msgr2=0x7f539c073ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:04.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.907+0000 7f53a08fd640 1 -- 172.21.15.92:0/1818897216 shutdown_connections 2024-04-22T07:04:04.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.907+0000 7f53a08fd640 1 -- 172.21.15.92:0/1818897216 wait complete. 2024-04-22T07:04:04.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.908+0000 7f53a08fd640 1 Processor -- start 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.908+0000 7f53a08fd640 1 -- start start 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f53a08fd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f539c0cb120 0x7f539c0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f53a08fd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c0d0b70 0x7f539c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f53a08fd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f539c0d0f50 0x7f539c1f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f53a08fd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0cae00 con 0x7f539c0d0b70 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f53a08fd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f539c15b690 con 0x7f539c0cb120 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f53a08fd640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f539c0caf80 con 0x7f539c0d0f50 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f5393fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c0d0b70 0x7f539c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f5393fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c0d0b70 0x7f539c15cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45078/0 (socket says 172.21.15.92:45078) 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f5393fff640 1 -- 172.21.15.92:0/862637776 learned_addr learned my addr 172.21.15.92:0/862637776 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f539ad9d640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f539c0d0f50 0x7f539c1f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f539a59c640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f539c0cb120 0x7f539c0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f5393fff640 1 -- 172.21.15.92:0/862637776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f539c0d0f50 msgr2=0x7f539c1f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f5393fff640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f539c0d0f50 0x7f539c1f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f5393fff640 1 -- 172.21.15.92:0/862637776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f539c0cb120 msgr2=0x7f539c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f5393fff640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f539c0cb120 0x7f539c0cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f5393fff640 1 -- 172.21.15.92:0/862637776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c151d70 con 0x7f539c0d0b70 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f5393fff640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c0d0b70 0x7f539c15cc70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f538402b7b0 tx=0x7f53840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f5392ffd640 1 -- 172.21.15.92:0/862637776 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384041020 con 0x7f539c0d0b70 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f5392ffd640 1 -- 172.21.15.92:0/862637776 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5384030cf0 con 0x7f539c0d0b70 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.909+0000 7f5392ffd640 1 -- 172.21.15.92:0/862637776 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384037040 con 0x7f539c0d0b70 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.910+0000 7f53a08fd640 1 -- 172.21.15.92:0/862637776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c1f0550 con 0x7f539c0d0b70 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.910+0000 7f53a08fd640 1 -- 172.21.15.92:0/862637776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c1f0b10 con 0x7f539c0d0b70 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.911+0000 7f53a08fd640 1 -- 172.21.15.92:0/862637776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c0cdae0 con 0x7f539c0d0b70 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.911+0000 7f5392ffd640 1 -- 172.21.15.92:0/862637776 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5384044020 con 0x7f539c0d0b70 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.911+0000 7f5392ffd640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f537c06c920 0x7f537c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:04.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.911+0000 7f5392ffd640 1 -- 172.21.15.92:0/862637776 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53840ad0d0 con 0x7f539c0d0b70 2024-04-22T07:04:04.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.911+0000 7f539a59c640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f537c06c920 0x7f537c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:04.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.912+0000 7f539a59c640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f537c06c920 0x7f537c06ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f538c0011e0 tx=0x7f538c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:04.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:04.916+0000 7f5392ffd640 1 -- 172.21.15.92:0/862637776 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f539c0cdae0 con 0x7f539c0d0b70 2024-04-22T07:04:04.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:04:04.955 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:04:05.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.132+0000 7f53a08fd640 1 -- 172.21.15.92:0/862637776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f539c069930 con 0x7f539c0d0b70 2024-04-22T07:04:05.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.795+0000 7f5392ffd640 1 -- 172.21.15.92:0/862637776 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v6) ==== 105+0+0 (secure 0 0 0) 0x7f538404b070 con 0x7f539c0d0b70 2024-04-22T07:04:05.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.797+0000 7f53a08fd640 1 -- 172.21.15.92:0/862637776 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f537c06c920 msgr2=0x7f537c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:05.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.797+0000 7f53a08fd640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f537c06c920 0x7f537c06ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f538c0011e0 tx=0x7f538c009040 comp rx=0 tx=0).stop 2024-04-22T07:04:05.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.797+0000 7f53a08fd640 1 -- 172.21.15.92:0/862637776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c0d0b70 msgr2=0x7f539c15cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:05.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.797+0000 7f53a08fd640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c0d0b70 0x7f539c15cc70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f538402b7b0 tx=0x7f53840046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:05.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.797+0000 7f53a08fd640 1 -- 172.21.15.92:0/862637776 shutdown_connections 2024-04-22T07:04:05.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.797+0000 7f53a08fd640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f537c06c920 0x7f537c06ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:05.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.797+0000 7f53a08fd640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f539c0d0f50 0x7f539c1f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:05.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.797+0000 7f53a08fd640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c0d0b70 0x7f539c15cc70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:05.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.797+0000 7f53a08fd640 1 --2- 172.21.15.92:0/862637776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f539c0cb120 0x7f539c0cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:05.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.797+0000 7f53a08fd640 1 -- 172.21.15.92:0/862637776 >> 172.21.15.92:0/862637776 conn(0x7f539c0736b0 msgr2=0x7f539c0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:05.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.797+0000 7f53a08fd640 1 -- 172.21.15.92:0/862637776 shutdown_connections 2024-04-22T07:04:05.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.797+0000 7f53a08fd640 1 -- 172.21.15.92:0/862637776 wait complete. 2024-04-22T07:04:05.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-22T07:04:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.916+0000 7febe6be6640 1 Processor -- start 2024-04-22T07:04:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.916+0000 7febe6be6640 1 -- start start 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febe6be6640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7febe0151d90 0x7febe00c5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febe6be6640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7febe006eed0 0x7febe00c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febe6be6640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe00c88b0 con 0x7febe00c6c20 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febe6be6640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7febe0152170 con 0x7febe006eed0 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febe6be6640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7febe00c7680 con 0x7febe0151d90 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febdffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7febe006eed0 0x7febe00c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febdffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7febe006eed0 0x7febe00c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45086/0 (socket says 172.21.15.92:45086) 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febdffff640 1 -- 172.21.15.92:0/79385965 learned_addr learned my addr 172.21.15.92:0/79385965 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febe4982640 1 --2- 172.21.15.92:0/79385965 >> v2:172.21.15.145:3300/0 conn(0x7febe0151d90 0x7febe00c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febdffff640 1 -- 172.21.15.92:0/79385965 >> v2:172.21.15.145:3300/0 conn(0x7febe0151d90 msgr2=0x7febe00c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febdffff640 1 --2- 172.21.15.92:0/79385965 >> v2:172.21.15.145:3300/0 conn(0x7febe0151d90 0x7febe00c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febdffff640 1 -- 172.21.15.92:0/79385965 >> v1:172.21.15.145:6789/0 conn(0x7febe00c6c20 legacy=0x7febe0157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:05.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febdffff640 1 -- 172.21.15.92:0/79385965 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe00c73a0 con 0x7febe006eed0 2024-04-22T07:04:05.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.917+0000 7febe4982640 1 --2- 172.21.15.92:0/79385965 >> v2:172.21.15.145:3300/0 conn(0x7febe0151d90 0x7febe00c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:05.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.918+0000 7febdffff640 1 --2- 172.21.15.92:0/79385965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe006eed0 0x7febe00c58a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7febe00712e0 tx=0x7febcc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63941f1b94f522f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:05.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.918+0000 7febdf7fe640 1 -- 172.21.15.92:0/79385965 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc034070 con 0x7febe006eed0 2024-04-22T07:04:05.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.918+0000 7febdf7fe640 1 -- 172.21.15.92:0/79385965 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febcc030860 con 0x7febe006eed0 2024-04-22T07:04:05.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.918+0000 7febdf7fe640 1 -- 172.21.15.92:0/79385965 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc030c30 con 0x7febe006eed0 2024-04-22T07:04:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.918+0000 7febe6be6640 1 -- 172.21.15.92:0/79385965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe006eed0 msgr2=0x7febe00c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.918+0000 7febe6be6640 1 --2- 172.21.15.92:0/79385965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe006eed0 0x7febe00c58a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7febe00712e0 tx=0x7febcc02b9a0 comp rx=0 tx=0).stop 2024-04-22T07:04:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.918+0000 7febe6be6640 1 -- 172.21.15.92:0/79385965 shutdown_connections 2024-04-22T07:04:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.918+0000 7febe6be6640 1 --2- 172.21.15.92:0/79385965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe006eed0 0x7febe00c58a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.918+0000 7febe6be6640 1 --2- 172.21.15.92:0/79385965 >> v2:172.21.15.145:3300/0 conn(0x7febe0151d90 0x7febe00c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.918+0000 7febe6be6640 1 -- 172.21.15.92:0/79385965 >> 172.21.15.92:0/79385965 conn(0x7febe00cc220 msgr2=0x7febe00cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.918+0000 7febe6be6640 1 -- 172.21.15.92:0/79385965 shutdown_connections 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.918+0000 7febe6be6640 1 -- 172.21.15.92:0/79385965 wait complete. 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.920+0000 7febe6be6640 1 Processor -- start 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.920+0000 7febe6be6640 1 -- start start 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.920+0000 7febe6be6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe006eed0 0x7febe01f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.920+0000 7febe6be6640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7febe00c6c20 0x7febe00c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.920+0000 7febe6be6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7febe0151d90 0x7febe0138de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.920+0000 7febe6be6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7febe00c3ba0 con 0x7febe006eed0 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.920+0000 7febe6be6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7febe00cd160 con 0x7febe0151d90 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.920+0000 7febe6be6640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7febe01582b0 con 0x7febe00c6c20 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.920+0000 7febe4982640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe006eed0 0x7febe01f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.920+0000 7febe4982640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe006eed0 0x7febe01f1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45092/0 (socket says 172.21.15.92:45092) 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.920+0000 7febe4982640 1 -- 172.21.15.92:0/226635575 learned_addr learned my addr 172.21.15.92:0/226635575 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:05.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febdffff640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7febe00c6c20 0x7febe00c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febe5183640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7febe0151d90 0x7febe0138de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febe4982640 1 -- 172.21.15.92:0/226635575 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7febe00c6c20 msgr2=0x7febe00c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febe4982640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7febe00c6c20 0x7febe00c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febe4982640 1 -- 172.21.15.92:0/226635575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7febe0151d90 msgr2=0x7febe0138de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febe4982640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7febe0151d90 0x7febe0138de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febe4982640 1 -- 172.21.15.92:0/226635575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe0147090 con 0x7febe006eed0 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febe5183640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7febe0151d90 0x7febe0138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febdffff640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7febe00c6c20 0x7febe00c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febe4982640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe006eed0 0x7febe01f1f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7febd400c7f0 tx=0x7febd400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febdd7fa640 1 -- 172.21.15.92:0/226635575 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4015070 con 0x7febe006eed0 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febdd7fa640 1 -- 172.21.15.92:0/226635575 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd40077c0 con 0x7febe006eed0 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febe6be6640 1 -- 172.21.15.92:0/226635575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe0155a00 con 0x7febe006eed0 2024-04-22T07:04:05.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febdd7fa640 1 -- 172.21.15.92:0/226635575 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4007b70 con 0x7febe006eed0 2024-04-22T07:04:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.921+0000 7febe6be6640 1 -- 172.21.15.92:0/226635575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe014c3e0 con 0x7febe006eed0 2024-04-22T07:04:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.922+0000 7febe6be6640 1 -- 172.21.15.92:0/226635575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe0069930 con 0x7febe006eed0 2024-04-22T07:04:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.923+0000 7febdd7fa640 1 -- 172.21.15.92:0/226635575 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febd4018070 con 0x7febe006eed0 2024-04-22T07:04:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.923+0000 7febdd7fa640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7febc406c920 0x7febc406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.923+0000 7febdd7fa640 1 -- 172.21.15.92:0/226635575 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7febd408bb50 con 0x7febe006eed0 2024-04-22T07:04:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.923+0000 7febdffff640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7febc406c920 0x7febc406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.923+0000 7febdffff640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7febc406c920 0x7febc406ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7febcc036000 tx=0x7febcc03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:05.927+0000 7febdd7fa640 1 -- 172.21.15.92:0/226635575 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febd405a010 con 0x7febe006eed0 2024-04-22T07:04:06.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.142+0000 7febe6be6640 1 -- 172.21.15.92:0/226635575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7febe00ccf40 con 0x7febe006eed0 2024-04-22T07:04:06.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.798+0000 7febdd7fa640 1 -- 172.21.15.92:0/226635575 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7febd4059910 con 0x7febe006eed0 2024-04-22T07:04:06.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.801+0000 7febe6be6640 1 -- 172.21.15.92:0/226635575 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7febc406c920 msgr2=0x7febc406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:06.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.801+0000 7febe6be6640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7febc406c920 0x7febc406ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7febcc036000 tx=0x7febcc03b040 comp rx=0 tx=0).stop 2024-04-22T07:04:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.801+0000 7febe6be6640 1 -- 172.21.15.92:0/226635575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe006eed0 msgr2=0x7febe01f1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.801+0000 7febe6be6640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe006eed0 0x7febe01f1f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7febd400c7f0 tx=0x7febd400ccb0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.801+0000 7febe6be6640 1 -- 172.21.15.92:0/226635575 shutdown_connections 2024-04-22T07:04:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.801+0000 7febe6be6640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7febc406c920 0x7febc406ede0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.801+0000 7febe6be6640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7febe0151d90 0x7febe0138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.801+0000 7febe6be6640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7febe00c6c20 0x7febe00c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.801+0000 7febe6be6640 1 --2- 172.21.15.92:0/226635575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febe006eed0 0x7febe01f1f30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.801+0000 7febe6be6640 1 -- 172.21.15.92:0/226635575 >> 172.21.15.92:0/226635575 conn(0x7febe00cc220 msgr2=0x7febe01450b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.801+0000 7febe6be6640 1 -- 172.21.15.92:0/226635575 shutdown_connections 2024-04-22T07:04:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.801+0000 7febe6be6640 1 -- 172.21.15.92:0/226635575 wait complete. 2024-04-22T07:04:06.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-22T07:04:06.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.917+0000 7ff396e43640 1 Processor -- start 2024-04-22T07:04:06.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.917+0000 7ff396e43640 1 -- start start 2024-04-22T07:04:06.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.917+0000 7ff396e43640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff3901552a0 0x7ff390070640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.917+0000 7ff396e43640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff390155680 con 0x7ff39006c8b0 2024-04-22T07:04:06.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.917+0000 7ff396e43640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3900c90b0 con 0x7ff39006b970 2024-04-22T07:04:06.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.917+0000 7ff396e43640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff3900c8e60 con 0x7ff3901552a0 2024-04-22T07:04:06.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff3953e0640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff39006c8b0 0x7ff390165780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39086/0 (socket says 172.21.15.92:39086) 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff3953e0640 1 -- 172.21.15.92:0/278014844 learned_addr learned my addr 172.21.15.92:0/278014844 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff394bdf640 1 --2- 172.21.15.92:0/278014844 >> v2:172.21.15.145:3300/0 conn(0x7ff3901552a0 0x7ff390070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 516546554 0 0) 0x7ff390155680 con 0x7ff39006c8b0 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff380003520 con 0x7ff39006c8b0 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3932902762 0 0) 0x7ff380003520 con 0x7ff39006c8b0 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 >> v2:172.21.15.145:3300/0 conn(0x7ff3901552a0 msgr2=0x7ff390070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 --2- 172.21.15.92:0/278014844 >> v2:172.21.15.145:3300/0 conn(0x7ff3901552a0 0x7ff390070640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 >> v1:172.21.15.145:6789/0 conn(0x7ff39006b970 legacy=0x7ff39006c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3900c9390 con 0x7ff39006c8b0 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7ff3880019f0 con 0x7ff39006c8b0 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 >> v1:172.21.15.92:6789/0 conn(0x7ff39006c8b0 legacy=0x7ff390165780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 --2- 172.21.15.92:0/278014844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3800039b0 0x7ff380003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 --2- 172.21.15.92:0/278014844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff380004800 0x7ff380004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 --2- 172.21.15.92:0/278014844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3800051c0 0x7ff3800141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3900c8e60 con 0x7ff3800051c0 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff380014880 con 0x7ff3800039b0 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff380014a80 con 0x7ff380004800 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff3953e0640 1 --2- 172.21.15.92:0/278014844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3800051c0 0x7ff3800141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff394bdf640 1 --2- 172.21.15.92:0/278014844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3800039b0 0x7ff380003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.918+0000 7ff38ffff640 1 --2- 172.21.15.92:0/278014844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff380004800 0x7ff380004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.919+0000 7ff3953e0640 1 -- 172.21.15.92:0/278014844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff380004800 msgr2=0x7ff380004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.919+0000 7ff3953e0640 1 --2- 172.21.15.92:0/278014844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff380004800 0x7ff380004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.919+0000 7ff3953e0640 1 -- 172.21.15.92:0/278014844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3800039b0 msgr2=0x7ff380003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.919+0000 7ff3953e0640 1 --2- 172.21.15.92:0/278014844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3800039b0 0x7ff380003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.919+0000 7ff3953e0640 1 -- 172.21.15.92:0/278014844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff380003520 con 0x7ff3800051c0 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.919+0000 7ff394bdf640 1 --2- 172.21.15.92:0/278014844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3800039b0 0x7ff380003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:06.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.919+0000 7ff3953e0640 1 -- 172.21.15.92:0/278014844 reap_dead start 2024-04-22T07:04:06.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.919+0000 7ff3953e0640 1 --2- 172.21.15.92:0/278014844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3800051c0 0x7ff3800141d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff38800a8d0 tx=0x7ff380005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89821903526a1c75 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:06.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.919+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384009690 con 0x7ff3800051c0 2024-04-22T07:04:06.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.919+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff388034650 con 0x7ff3800051c0 2024-04-22T07:04:06.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.919+0000 7ff38f7fe640 1 -- 172.21.15.92:0/278014844 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38800a4a0 con 0x7ff3800051c0 2024-04-22T07:04:06.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.920+0000 7ff396e43640 1 -- 172.21.15.92:0/278014844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3800051c0 msgr2=0x7ff3800141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:06.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.920+0000 7ff396e43640 1 --2- 172.21.15.92:0/278014844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3800051c0 0x7ff3800141d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff38800a8d0 tx=0x7ff380005830 comp rx=0 tx=0).stop 2024-04-22T07:04:06.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.920+0000 7ff396e43640 1 -- 172.21.15.92:0/278014844 shutdown_connections 2024-04-22T07:04:06.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.920+0000 7ff396e43640 1 --2- 172.21.15.92:0/278014844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3800051c0 0x7ff3800141d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.920+0000 7ff396e43640 1 -- 172.21.15.92:0/278014844 >> 172.21.15.92:0/278014844 conn(0x7ff390071640 msgr2=0x7ff390071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:06.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.920+0000 7ff396e43640 1 -- 172.21.15.92:0/278014844 shutdown_connections 2024-04-22T07:04:06.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.920+0000 7ff396e43640 1 -- 172.21.15.92:0/278014844 wait complete. 2024-04-22T07:04:06.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.921+0000 7ff396e43640 1 Processor -- start 2024-04-22T07:04:06.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.921+0000 7ff396e43640 1 -- start start 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff396e43640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800051c0 0x7ff390155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff396e43640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff390157a40 0x7ff390158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff396e43640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3900ca740 0x7ff3901e0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff396e43640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff39006fb60 con 0x7ff390157a40 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff396e43640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3900701b0 con 0x7ff3900ca740 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff396e43640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff39006fe40 con 0x7ff3800051c0 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff38ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff390157a40 0x7ff390158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff38ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff390157a40 0x7ff390158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45120/0 (socket says 172.21.15.92:45120) 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff38ffff640 1 -- 172.21.15.92:0/2046020879 learned_addr learned my addr 172.21.15.92:0/2046020879 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff394bdf640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800051c0 0x7ff390155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff3953e0640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3900ca740 0x7ff3901e0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff38ffff640 1 -- 172.21.15.92:0/2046020879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800051c0 msgr2=0x7ff390155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff38ffff640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800051c0 0x7ff390155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff38ffff640 1 -- 172.21.15.92:0/2046020879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3900ca740 msgr2=0x7ff3901e0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff38ffff640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3900ca740 0x7ff3901e0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff38ffff640 1 -- 172.21.15.92:0/2046020879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3901e2660 con 0x7ff390157a40 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff394bdf640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800051c0 0x7ff390155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.922+0000 7ff3953e0640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3900ca740 0x7ff3901e0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.923+0000 7ff38ffff640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff390157a40 0x7ff390158440 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00f620 tx=0x7ff37c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.923+0000 7ff38d7fa640 1 -- 172.21.15.92:0/2046020879 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c0052e0 con 0x7ff390157a40 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.923+0000 7ff38d7fa640 1 -- 172.21.15.92:0/2046020879 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff37c018070 con 0x7ff390157a40 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.923+0000 7ff396e43640 1 -- 172.21.15.92:0/2046020879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3901e3ae0 con 0x7ff390157a40 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.923+0000 7ff38d7fa640 1 -- 172.21.15.92:0/2046020879 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c003810 con 0x7ff390157a40 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.923+0000 7ff396e43640 1 -- 172.21.15.92:0/2046020879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3901e2010 con 0x7ff390157a40 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.924+0000 7ff396e43640 1 -- 172.21.15.92:0/2046020879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff390068b00 con 0x7ff390157a40 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.924+0000 7ff38d7fa640 1 -- 172.21.15.92:0/2046020879 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff37c0039b0 con 0x7ff390157a40 2024-04-22T07:04:06.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.924+0000 7ff38d7fa640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff37406c8f0 0x7ff37406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.925+0000 7ff394bdf640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff37406c8f0 0x7ff37406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.925+0000 7ff38d7fa640 1 -- 172.21.15.92:0/2046020879 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff37c08b900 con 0x7ff390157a40 2024-04-22T07:04:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.925+0000 7ff394bdf640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff37406c8f0 0x7ff37406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff384009b40 tx=0x7ff384008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:06.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:06.929+0000 7ff38d7fa640 1 -- 172.21.15.92:0/2046020879 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff37c059d30 con 0x7ff390157a40 2024-04-22T07:04:07.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.142+0000 7ff396e43640 1 -- 172.21.15.92:0/2046020879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7ff390158160 con 0x7ff390157a40 2024-04-22T07:04:07.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.799+0000 7ff38d7fa640 1 -- 172.21.15.92:0/2046020879 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) ==== 115+0+0 (secure 0 0 0) 0x7ff37c059630 con 0x7ff390157a40 2024-04-22T07:04:07.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.802+0000 7ff396e43640 1 -- 172.21.15.92:0/2046020879 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff37406c8f0 msgr2=0x7ff37406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.802+0000 7ff396e43640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff37406c8f0 0x7ff37406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff384009b40 tx=0x7ff384008040 comp rx=0 tx=0).stop 2024-04-22T07:04:07.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.802+0000 7ff396e43640 1 -- 172.21.15.92:0/2046020879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff390157a40 msgr2=0x7ff390158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.802+0000 7ff396e43640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff390157a40 0x7ff390158440 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00f620 tx=0x7ff37c00fae0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.802+0000 7ff396e43640 1 -- 172.21.15.92:0/2046020879 shutdown_connections 2024-04-22T07:04:07.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.802+0000 7ff396e43640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff37406c8f0 0x7ff37406edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.802+0000 7ff396e43640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3900ca740 0x7ff3901e0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.802+0000 7ff396e43640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff390157a40 0x7ff390158440 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.802+0000 7ff396e43640 1 --2- 172.21.15.92:0/2046020879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800051c0 0x7ff390155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.802+0000 7ff396e43640 1 -- 172.21.15.92:0/2046020879 >> 172.21.15.92:0/2046020879 conn(0x7ff390071640 msgr2=0x7ff39014fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:07.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.802+0000 7ff396e43640 1 -- 172.21.15.92:0/2046020879 shutdown_connections 2024-04-22T07:04:07.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.802+0000 7ff396e43640 1 -- 172.21.15.92:0/2046020879 wait complete. 2024-04-22T07:04:07.814 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-22T07:04:07.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.916+0000 7fcff32b3640 1 Processor -- start 2024-04-22T07:04:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.916+0000 7fcff32b3640 1 -- start start 2024-04-22T07:04:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.917+0000 7fcff32b3640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcfec154fe0 0x7fcfec15b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.917+0000 7fcff32b3640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfec15ae70 con 0x7fcfec06c6a0 2024-04-22T07:04:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.917+0000 7fcff32b3640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfec0c6240 con 0x7fcfec0c5940 2024-04-22T07:04:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.917+0000 7fcff32b3640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcfec0c6520 con 0x7fcfec154fe0 2024-04-22T07:04:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.917+0000 7fcff084e640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcfec06c6a0 0x7fcfec0728b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46570/0 (socket says 172.21.15.92:46570) 2024-04-22T07:04:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.917+0000 7fcff084e640 1 -- 172.21.15.92:0/141509446 learned_addr learned my addr 172.21.15.92:0/141509446 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.917+0000 7fcff104f640 1 --2- 172.21.15.92:0/141509446 >> v2:172.21.15.145:3300/0 conn(0x7fcfec154fe0 0x7fcfec15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.917+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 359579103 0 0) 0x7fcfec15ae70 con 0x7fcfec06c6a0 2024-04-22T07:04:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.917+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfd8003520 con 0x7fcfec06c6a0 2024-04-22T07:04:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3713198208 0 0) 0x7fcfd8003520 con 0x7fcfec06c6a0 2024-04-22T07:04:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 >> v2:172.21.15.145:3300/0 conn(0x7fcfec154fe0 msgr2=0x7fcfec15b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 --2- 172.21.15.92:0/141509446 >> v2:172.21.15.145:3300/0 conn(0x7fcfec154fe0 0x7fcfec15b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 >> v1:172.21.15.145:6789/0 conn(0x7fcfec0c5940 legacy=0x7fcfec165640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfec15b110 con 0x7fcfec06c6a0 2024-04-22T07:04:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7fcfdc001670 con 0x7fcfec06c6a0 2024-04-22T07:04:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 >> v1:172.21.15.92:6789/0 conn(0x7fcfec06c6a0 legacy=0x7fcfec0728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 --2- 172.21.15.92:0/141509446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd80039b0 0x7fcfd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 --2- 172.21.15.92:0/141509446 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfd8004800 0x7fcfd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 --2- 172.21.15.92:0/141509446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfd80051c0 0x7fcfd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec0c6520 con 0x7fcfd80051c0 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd800c070 con 0x7fcfd80039b0 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd800c270 con 0x7fcfd8004800 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcff104f640 1 --2- 172.21.15.92:0/141509446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd80039b0 0x7fcfd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcff1850640 1 --2- 172.21.15.92:0/141509446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfd80051c0 0x7fcfd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcff084e640 1 --2- 172.21.15.92:0/141509446 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfd8004800 0x7fcfd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcff1850640 1 -- 172.21.15.92:0/141509446 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfd8004800 msgr2=0x7fcfd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcff1850640 1 --2- 172.21.15.92:0/141509446 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfd8004800 0x7fcfd8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcff1850640 1 -- 172.21.15.92:0/141509446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd80039b0 msgr2=0x7fcfd8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcff1850640 1 --2- 172.21.15.92:0/141509446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd80039b0 0x7fcfd8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.918+0000 7fcff1850640 1 -- 172.21.15.92:0/141509446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd8003520 con 0x7fcfd80051c0 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.919+0000 7fcff1850640 1 -- 172.21.15.92:0/141509446 reap_dead start 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.919+0000 7fcff1850640 1 --2- 172.21.15.92:0/141509446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfd80051c0 0x7fcfd800b9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4001750 tx=0x7fcfd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d48ca5a9699ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.919+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4001370 con 0x7fcfd80051c0 2024-04-22T07:04:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.919+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe4005310 con 0x7fcfd80051c0 2024-04-22T07:04:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.919+0000 7fcfe3fff640 1 -- 172.21.15.92:0/141509446 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4003830 con 0x7fcfd80051c0 2024-04-22T07:04:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.919+0000 7fcff32b3640 1 -- 172.21.15.92:0/141509446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfd80051c0 msgr2=0x7fcfd800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.919+0000 7fcff32b3640 1 --2- 172.21.15.92:0/141509446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfd80051c0 0x7fcfd800b9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4001750 tx=0x7fcfd8005830 comp rx=0 tx=0).stop 2024-04-22T07:04:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.919+0000 7fcff32b3640 1 -- 172.21.15.92:0/141509446 shutdown_connections 2024-04-22T07:04:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.919+0000 7fcff32b3640 1 --2- 172.21.15.92:0/141509446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfd80051c0 0x7fcfd800b9c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.919+0000 7fcff32b3640 1 -- 172.21.15.92:0/141509446 >> 172.21.15.92:0/141509446 conn(0x7fcfec067890 msgr2=0x7fcfec073770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.920+0000 7fcff32b3640 1 -- 172.21.15.92:0/141509446 shutdown_connections 2024-04-22T07:04:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.920+0000 7fcff32b3640 1 -- 172.21.15.92:0/141509446 wait complete. 2024-04-22T07:04:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.921+0000 7fcff32b3640 1 Processor -- start 2024-04-22T07:04:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.921+0000 7fcff32b3640 1 -- start start 2024-04-22T07:04:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.921+0000 7fcff32b3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd80051c0 0x7fcfec1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.921+0000 7fcff32b3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfec0cd9d0 0x7fcfec1f1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.921+0000 7fcff32b3640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfec157b20 0x7fcfec14af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.921+0000 7fcff32b3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1553c0 con 0x7fcfec0cd9d0 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.921+0000 7fcff32b3640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec0c5e90 con 0x7fcfd80051c0 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.921+0000 7fcff32b3640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcfec1f0f90 con 0x7fcfec157b20 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.921+0000 7fcff084e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfec0cd9d0 0x7fcfec1f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.921+0000 7fcff084e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfec0cd9d0 0x7fcfec1f1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35910/0 (socket says 172.21.15.92:35910) 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.921+0000 7fcff084e640 1 -- 172.21.15.92:0/3684226103 learned_addr learned my addr 172.21.15.92:0/3684226103 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcff1850640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfec157b20 0x7fcfec14af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcff104f640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd80051c0 0x7fcfec1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcff084e640 1 -- 172.21.15.92:0/3684226103 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfec157b20 msgr2=0x7fcfec14af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcff084e640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfec157b20 0x7fcfec14af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcff084e640 1 -- 172.21.15.92:0/3684226103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd80051c0 msgr2=0x7fcfec1571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcff084e640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd80051c0 0x7fcfec1571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcff084e640 1 -- 172.21.15.92:0/3684226103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfec155d30 con 0x7fcfec0cd9d0 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcff084e640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfec0cd9d0 0x7fcfec1f1420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc004fa0 tx=0x7fcfdc0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcfe1ffb640 1 -- 172.21.15.92:0/3684226103 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc007740 con 0x7fcfec0cd9d0 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcfe1ffb640 1 -- 172.21.15.92:0/3684226103 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfdc018070 con 0x7fcfec0cd9d0 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcff32b3640 1 -- 172.21.15.92:0/3684226103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfec155770 con 0x7fcfec0cd9d0 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcfe1ffb640 1 -- 172.21.15.92:0/3684226103 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc00fbb0 con 0x7fcfec0cd9d0 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.922+0000 7fcff32b3640 1 -- 172.21.15.92:0/3684226103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfec1f0c20 con 0x7fcfec0cd9d0 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.923+0000 7fcff32b3640 1 -- 172.21.15.92:0/3684226103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfec069930 con 0x7fcfec0cd9d0 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.924+0000 7fcfe1ffb640 1 -- 172.21.15.92:0/3684226103 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfdc00fd50 con 0x7fcfec0cd9d0 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.924+0000 7fcfe1ffb640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcfd006cac0 0x7fcfd006ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.924+0000 7fcfe1ffb640 1 -- 172.21.15.92:0/3684226103 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcfdc08bb30 con 0x7fcfec0cd9d0 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.924+0000 7fcff104f640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcfd006cac0 0x7fcfd006ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.924+0000 7fcff104f640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcfd006cac0 0x7fcfd006ef80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009fd0 tx=0x7fcfe800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:07.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:07.928+0000 7fcfe1ffb640 1 -- 172.21.15.92:0/3684226103 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfdc059f60 con 0x7fcfec0cd9d0 2024-04-22T07:04:08.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.140+0000 7fcff32b3640 1 -- 172.21.15.92:0/3684226103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfec0ca920 con 0x7fcfec0cd9d0 2024-04-22T07:04:08.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.141+0000 7fcfe1ffb640 1 -- 172.21.15.92:0/3684226103 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6825 (secure 0 0 0) 0x7fcfdc059860 con 0x7fcfec0cd9d0 2024-04-22T07:04:08.139 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:08.139 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1059009649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1059009649},{"type":"v1","addr":"172.21.15.145:6835","nonce":1059009649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6836/1058270486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1058270486},{"type":"v1","addr":"172.21.15.145:6836","nonce":1058270486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/3258302920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3258302920},{"type":"v1","addr":"172.21.15.145:6837","nonce":3258302920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/101798872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":101798872},{"type":"v1","addr":"172.21.15.92:6837","nonce":101798872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:03.129095+0000","modified":"2024-04-22T07:04:07.143345+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.92:6839/3493022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3493022212},{"type":"v1","addr":"172.21.15.92:6839","nonce":3493022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T07:04:08.140 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2024-04-22T07:04:08.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.143+0000 7fcff32b3640 1 -- 172.21.15.92:0/3684226103 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcfd006cac0 msgr2=0x7fcfd006ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.143+0000 7fcff32b3640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcfd006cac0 0x7fcfd006ef80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009fd0 tx=0x7fcfe800b040 comp rx=0 tx=0).stop 2024-04-22T07:04:08.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.143+0000 7fcff32b3640 1 -- 172.21.15.92:0/3684226103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfec0cd9d0 msgr2=0x7fcfec1f1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.143+0000 7fcff32b3640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfec0cd9d0 0x7fcfec1f1420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc004fa0 tx=0x7fcfdc0074f0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.143+0000 7fcff32b3640 1 -- 172.21.15.92:0/3684226103 shutdown_connections 2024-04-22T07:04:08.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.143+0000 7fcff32b3640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcfd006cac0 0x7fcfd006ef80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.143+0000 7fcff32b3640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfec157b20 0x7fcfec14af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.143+0000 7fcff32b3640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfec0cd9d0 0x7fcfec1f1420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.143+0000 7fcff32b3640 1 --2- 172.21.15.92:0/3684226103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd80051c0 0x7fcfec1571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.143+0000 7fcff32b3640 1 -- 172.21.15.92:0/3684226103 >> 172.21.15.92:0/3684226103 conn(0x7fcfec067890 msgr2=0x7fcfec0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:08.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.143+0000 7fcff32b3640 1 -- 172.21.15.92:0/3684226103 shutdown_connections 2024-04-22T07:04:08.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.143+0000 7fcff32b3640 1 -- 172.21.15.92:0/3684226103 wait complete. 2024-04-22T07:04:08.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:08.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.250+0000 7faccc7a4640 1 Processor -- start 2024-04-22T07:04:08.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.250+0000 7faccc7a4640 1 -- start start 2024-04-22T07:04:08.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7faccc7a4640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7facc41561b0 0x7facc40c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7faccc7a4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7facc4070e80 0x7facc40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7faccc7a4640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7facc40c5210 0x7facc40ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7faccc7a4640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7facc40704a0 con 0x7facc41561b0 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7faccc7a4640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7facc4159510 con 0x7facc40c5210 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7faccc7a4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7facc41597f0 con 0x7facc4070e80 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7facca540640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7facc41561b0 0x7facc40c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7facca540640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7facc41561b0 0x7facc40c3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35926/0 (socket says 172.21.15.92:35926) 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7facca540640 1 -- 172.21.15.92:0/1176620709 learned_addr learned my addr 172.21.15.92:0/1176620709 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7facca540640 1 -- 172.21.15.92:0/1176620709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7facc4070e80 msgr2=0x7facc40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7facca540640 1 --2- 172.21.15.92:0/1176620709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7facc4070e80 0x7facc40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7facca540640 1 -- 172.21.15.92:0/1176620709 >> v2:172.21.15.145:3300/0 conn(0x7facc40c5210 msgr2=0x7facc40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7facca540640 1 --2- 172.21.15.92:0/1176620709 >> v2:172.21.15.145:3300/0 conn(0x7facc40c5210 0x7facc40ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.251+0000 7facca540640 1 -- 172.21.15.92:0/1176620709 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc40c8f70 con 0x7facc41561b0 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.252+0000 7facca540640 1 --2- 172.21.15.92:0/1176620709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc41561b0 0x7facc40c3410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7facc40c9f70 tx=0x7facc002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7223f0ff62e33aa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.252+0000 7facc953e640 1 -- 172.21.15.92:0/1176620709 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0034070 con 0x7facc41561b0 2024-04-22T07:04:08.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.252+0000 7facc953e640 1 -- 172.21.15.92:0/1176620709 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc0030860 con 0x7facc41561b0 2024-04-22T07:04:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.252+0000 7facc953e640 1 -- 172.21.15.92:0/1176620709 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0030c30 con 0x7facc41561b0 2024-04-22T07:04:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.252+0000 7faccc7a4640 1 -- 172.21.15.92:0/1176620709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc41561b0 msgr2=0x7facc40c3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.252+0000 7faccc7a4640 1 --2- 172.21.15.92:0/1176620709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc41561b0 0x7facc40c3410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7facc40c9f70 tx=0x7facc002b9a0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.252+0000 7faccc7a4640 1 -- 172.21.15.92:0/1176620709 shutdown_connections 2024-04-22T07:04:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.252+0000 7faccc7a4640 1 --2- 172.21.15.92:0/1176620709 >> v2:172.21.15.145:3300/0 conn(0x7facc40c5210 0x7facc40ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.252+0000 7faccc7a4640 1 --2- 172.21.15.92:0/1176620709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7facc4070e80 0x7facc40ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.252+0000 7faccc7a4640 1 --2- 172.21.15.92:0/1176620709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc41561b0 0x7facc40c3410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.252+0000 7faccc7a4640 1 -- 172.21.15.92:0/1176620709 >> 172.21.15.92:0/1176620709 conn(0x7facc406db80 msgr2=0x7facc406df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.253+0000 7faccc7a4640 1 -- 172.21.15.92:0/1176620709 shutdown_connections 2024-04-22T07:04:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.253+0000 7faccc7a4640 1 -- 172.21.15.92:0/1176620709 wait complete. 2024-04-22T07:04:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.254+0000 7faccc7a4640 1 Processor -- start 2024-04-22T07:04:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.254+0000 7faccc7a4640 1 -- start start 2024-04-22T07:04:08.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.254+0000 7faccc7a4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4070e80 0x7facc41deb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.254+0000 7faccc7a4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7facc40c5210 0x7facc4153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.254+0000 7faccc7a4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc41561b0 0x7facc41ee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.254+0000 7faccc7a4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7facc4156590 con 0x7facc41561b0 2024-04-22T07:04:08.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.254+0000 7faccc7a4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7facc40c97b0 con 0x7facc4070e80 2024-04-22T07:04:08.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.254+0000 7faccc7a4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7facc4070930 con 0x7facc40c5210 2024-04-22T07:04:08.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.254+0000 7faccad41640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc41561b0 0x7facc41ee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.254+0000 7faccad41640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc41561b0 0x7facc41ee5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35940/0 (socket says 172.21.15.92:35940) 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.254+0000 7faccad41640 1 -- 172.21.15.92:0/792937320 learned_addr learned my addr 172.21.15.92:0/792937320 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7facc9d3f640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7facc40c5210 0x7facc4153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7facca540640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4070e80 0x7facc41deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7faccad41640 1 -- 172.21.15.92:0/792937320 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7facc40c5210 msgr2=0x7facc4153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7faccad41640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7facc40c5210 0x7facc4153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7faccad41640 1 -- 172.21.15.92:0/792937320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4070e80 msgr2=0x7facc41deb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7faccad41640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4070e80 0x7facc41deb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7faccad41640 1 -- 172.21.15.92:0/792937320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc41f1c00 con 0x7facc41561b0 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7facc9d3f640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7facc40c5210 0x7facc4153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7facca540640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4070e80 0x7facc41deb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7faccad41640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc41561b0 0x7facc41ee5f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7facbc00ba10 tx=0x7facbc00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:08.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7facbaffd640 1 -- 172.21.15.92:0/792937320 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc019070 con 0x7facc41561b0 2024-04-22T07:04:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7facbaffd640 1 -- 172.21.15.92:0/792937320 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facbc00cae0 con 0x7facc41561b0 2024-04-22T07:04:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7faccc7a4640 1 -- 172.21.15.92:0/792937320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc41df350 con 0x7facc41561b0 2024-04-22T07:04:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7facbaffd640 1 -- 172.21.15.92:0/792937320 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc014440 con 0x7facc41561b0 2024-04-22T07:04:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.255+0000 7faccc7a4640 1 -- 172.21.15.92:0/792937320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc41f2180 con 0x7facc41561b0 2024-04-22T07:04:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.256+0000 7faccc7a4640 1 -- 172.21.15.92:0/792937320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc4069930 con 0x7facc41561b0 2024-04-22T07:04:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.256+0000 7facbaffd640 1 -- 172.21.15.92:0/792937320 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facbc01c070 con 0x7facc41561b0 2024-04-22T07:04:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.257+0000 7facbaffd640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7faca806c920 0x7faca806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.257+0000 7facbaffd640 1 -- 172.21.15.92:0/792937320 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7facbc08b770 con 0x7facc41561b0 2024-04-22T07:04:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.257+0000 7facca540640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7faca806c920 0x7faca806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.257+0000 7facca540640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7faca806c920 0x7faca806ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7facc0036000 tx=0x7facc003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:08.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.261+0000 7facbaffd640 1 -- 172.21.15.92:0/792937320 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7facbc059c30 con 0x7facc41561b0 2024-04-22T07:04:08.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.435+0000 7faccc7a4640 1 -- 172.21.15.92:0/792937320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facc4070c60 con 0x7facc41561b0 2024-04-22T07:04:08.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.437+0000 7facbaffd640 1 -- 172.21.15.92:0/792937320 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7facbc059530 con 0x7facc41561b0 2024-04-22T07:04:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:08.435 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:04:04.784969+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:08.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.438+0000 7faccc7a4640 1 -- 172.21.15.92:0/792937320 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7faca806c920 msgr2=0x7faca806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.438+0000 7faccc7a4640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7faca806c920 0x7faca806ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7facc0036000 tx=0x7facc003b040 comp rx=0 tx=0).stop 2024-04-22T07:04:08.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.438+0000 7faccc7a4640 1 -- 172.21.15.92:0/792937320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc41561b0 msgr2=0x7facc41ee5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.438+0000 7faccc7a4640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc41561b0 0x7facc41ee5f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7facbc00ba10 tx=0x7facbc00bed0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.438+0000 7faccc7a4640 1 -- 172.21.15.92:0/792937320 shutdown_connections 2024-04-22T07:04:08.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.438+0000 7faccc7a4640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7faca806c920 0x7faca806ede0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.438+0000 7faccc7a4640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facc41561b0 0x7facc41ee5f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.438+0000 7faccc7a4640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7facc40c5210 0x7facc4153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.438+0000 7faccc7a4640 1 --2- 172.21.15.92:0/792937320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc4070e80 0x7facc41deb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.438+0000 7faccc7a4640 1 -- 172.21.15.92:0/792937320 >> 172.21.15.92:0/792937320 conn(0x7facc406db80 msgr2=0x7facc4154550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:08.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.439+0000 7faccc7a4640 1 -- 172.21.15.92:0/792937320 shutdown_connections 2024-04-22T07:04:08.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.439+0000 7faccc7a4640 1 -- 172.21.15.92:0/792937320 wait complete. 2024-04-22T07:04:08.447 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-22T07:04:08.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:08.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.546+0000 7fda8c2a0640 1 Processor -- start 2024-04-22T07:04:08.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.546+0000 7fda8c2a0640 1 -- start start 2024-04-22T07:04:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8c2a0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda840cd200 0x7fda8415b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8c2a0640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fda840cd5e0 0x7fda84154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8c2a0640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda8415e660 con 0x7fda840cb6b0 2024-04-22T07:04:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8c2a0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fda8415f480 con 0x7fda840cd200 2024-04-22T07:04:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8c2a0640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fda8415e900 con 0x7fda840cd5e0 2024-04-22T07:04:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8a03c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda840cd200 0x7fda8415b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8a03c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda840cd200 0x7fda8415b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35942/0 (socket says 172.21.15.92:35942) 2024-04-22T07:04:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8a03c640 1 -- 172.21.15.92:0/3938579552 learned_addr learned my addr 172.21.15.92:0/3938579552 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8983b640 1 --2- 172.21.15.92:0/3938579552 >> v2:172.21.15.145:3300/0 conn(0x7fda840cd5e0 0x7fda84154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8a03c640 1 -- 172.21.15.92:0/3938579552 >> v2:172.21.15.145:3300/0 conn(0x7fda840cd5e0 msgr2=0x7fda84154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8a03c640 1 --2- 172.21.15.92:0/3938579552 >> v2:172.21.15.145:3300/0 conn(0x7fda840cd5e0 0x7fda84154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8a03c640 1 -- 172.21.15.92:0/3938579552 >> v1:172.21.15.145:6789/0 conn(0x7fda840cb6b0 legacy=0x7fda840b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:08.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8a03c640 1 -- 172.21.15.92:0/3938579552 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8415ebe0 con 0x7fda840cd200 2024-04-22T07:04:08.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.547+0000 7fda8983b640 1 --2- 172.21.15.92:0/3938579552 >> v2:172.21.15.145:3300/0 conn(0x7fda840cd5e0 0x7fda84154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:08.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.548+0000 7fda8a03c640 1 --2- 172.21.15.92:0/3938579552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda840cd200 0x7fda8415b9b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fda840741b0 tx=0x7fda8002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a501c07197fe6bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:08.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.548+0000 7fda8903a640 1 -- 172.21.15.92:0/3938579552 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8002caf0 con 0x7fda840cd200 2024-04-22T07:04:08.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.548+0000 7fda8903a640 1 -- 172.21.15.92:0/3938579552 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda8002cc90 con 0x7fda840cd200 2024-04-22T07:04:08.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.548+0000 7fda8903a640 1 -- 172.21.15.92:0/3938579552 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80034610 con 0x7fda840cd200 2024-04-22T07:04:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.548+0000 7fda8c2a0640 1 -- 172.21.15.92:0/3938579552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda840cd200 msgr2=0x7fda8415b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.548+0000 7fda8c2a0640 1 --2- 172.21.15.92:0/3938579552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda840cd200 0x7fda8415b9b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fda840741b0 tx=0x7fda8002bf50 comp rx=0 tx=0).stop 2024-04-22T07:04:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.548+0000 7fda8c2a0640 1 -- 172.21.15.92:0/3938579552 shutdown_connections 2024-04-22T07:04:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.548+0000 7fda8c2a0640 1 --2- 172.21.15.92:0/3938579552 >> v2:172.21.15.145:3300/0 conn(0x7fda840cd5e0 0x7fda84154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.548+0000 7fda8c2a0640 1 --2- 172.21.15.92:0/3938579552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda840cd200 0x7fda8415b9b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.548+0000 7fda8c2a0640 1 -- 172.21.15.92:0/3938579552 >> 172.21.15.92:0/3938579552 conn(0x7fda84073bc0 msgr2=0x7fda840728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.548+0000 7fda8c2a0640 1 -- 172.21.15.92:0/3938579552 shutdown_connections 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.548+0000 7fda8c2a0640 1 -- 172.21.15.92:0/3938579552 wait complete. 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.549+0000 7fda8c2a0640 1 Processor -- start 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.550+0000 7fda8c2a0640 1 -- start start 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.550+0000 7fda8c2a0640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda840cb6b0 0x7fda840d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.550+0000 7fda8c2a0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda840cd200 0x7fda840c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.550+0000 7fda8c2a0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda840cd5e0 0x7fda8415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.550+0000 7fda8c2a0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fda840cc430 con 0x7fda840cd5e0 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.550+0000 7fda8c2a0640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fda8415f1a0 con 0x7fda840cd200 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.550+0000 7fda8c2a0640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fda840cc5b0 con 0x7fda840cb6b0 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.550+0000 7fda8a83d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda840cd5e0 0x7fda8415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.550+0000 7fda8a83d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda840cd5e0 0x7fda8415a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35950/0 (socket says 172.21.15.92:35950) 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.550+0000 7fda8a83d640 1 -- 172.21.15.92:0/4262671705 learned_addr learned my addr 172.21.15.92:0/4262671705 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.550+0000 7fda8983b640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda840cd200 0x7fda840c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.550+0000 7fda8a03c640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda840cb6b0 0x7fda840d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.551+0000 7fda8a83d640 1 -- 172.21.15.92:0/4262671705 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda840cb6b0 msgr2=0x7fda840d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.551+0000 7fda8a83d640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda840cb6b0 0x7fda840d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.551+0000 7fda8a83d640 1 -- 172.21.15.92:0/4262671705 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda840cd200 msgr2=0x7fda840c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.551+0000 7fda8a83d640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda840cd200 0x7fda840c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.551+0000 7fda8a83d640 1 -- 172.21.15.92:0/4262671705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda840c96d0 con 0x7fda840cd5e0 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.551+0000 7fda8a83d640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda840cd5e0 0x7fda8415a730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00c440 tx=0x7fda7c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.551+0000 7fda7affd640 1 -- 172.21.15.92:0/4262671705 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c011790 con 0x7fda840cd5e0 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.551+0000 7fda7affd640 1 -- 172.21.15.92:0/4262671705 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7c017070 con 0x7fda840cd5e0 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.551+0000 7fda8c2a0640 1 -- 172.21.15.92:0/4262671705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda840c99b0 con 0x7fda840cd5e0 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.551+0000 7fda7affd640 1 -- 172.21.15.92:0/4262671705 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c011b70 con 0x7fda840cd5e0 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.551+0000 7fda8c2a0640 1 -- 172.21.15.92:0/4262671705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda840c9f70 con 0x7fda840cd5e0 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.552+0000 7fda8c2a0640 1 -- 172.21.15.92:0/4262671705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda840c74b0 con 0x7fda840cd5e0 2024-04-22T07:04:08.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.552+0000 7fda7affd640 1 -- 172.21.15.92:0/4262671705 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda7c011d10 con 0x7fda840cd5e0 2024-04-22T07:04:08.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.553+0000 7fda7affd640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fda6806c8f0 0x7fda6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.553+0000 7fda8a03c640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fda6806c8f0 0x7fda6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.553+0000 7fda7affd640 1 -- 172.21.15.92:0/4262671705 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fda7c08b810 con 0x7fda840cd5e0 2024-04-22T07:04:08.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.553+0000 7fda8a03c640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fda6806c8f0 0x7fda6806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda80035000 tx=0x7fda80030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:08.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.557+0000 7fda7affd640 1 -- 172.21.15.92:0/4262671705 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda840c74b0 con 0x7fda840cd5e0 2024-04-22T07:04:08.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.728+0000 7fda8c2a0640 1 -- 172.21.15.92:0/4262671705 --> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda8406c800 con 0x7fda6806c8f0 2024-04-22T07:04:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.734+0000 7fda7affd640 1 -- 172.21.15.92:0/4262671705 <== mgr.4107 v2:172.21.15.92:6832/479317737 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251851 (secure 0 0 0) 0x7fda8406c800 con 0x7fda6806c8f0 2024-04-22T07:04:08.735 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:08.736 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-22T07:04:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.737+0000 7fda8c2a0640 1 -- 172.21.15.92:0/4262671705 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fda6806c8f0 msgr2=0x7fda6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.737+0000 7fda8c2a0640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fda6806c8f0 0x7fda6806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda80035000 tx=0x7fda80030000 comp rx=0 tx=0).stop 2024-04-22T07:04:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.737+0000 7fda8c2a0640 1 -- 172.21.15.92:0/4262671705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda840cd5e0 msgr2=0x7fda8415a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.737+0000 7fda8c2a0640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda840cd5e0 0x7fda8415a730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00c440 tx=0x7fda7c00c900 comp rx=0 tx=0).stop 2024-04-22T07:04:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.737+0000 7fda8c2a0640 1 -- 172.21.15.92:0/4262671705 shutdown_connections 2024-04-22T07:04:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.737+0000 7fda8c2a0640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fda6806c8f0 0x7fda6806edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.737+0000 7fda8c2a0640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda840cd5e0 0x7fda8415a730 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.737+0000 7fda8c2a0640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda840cd200 0x7fda840c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.737+0000 7fda8c2a0640 1 --2- 172.21.15.92:0/4262671705 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda840cb6b0 0x7fda840d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.737+0000 7fda8c2a0640 1 -- 172.21.15.92:0/4262671705 >> 172.21.15.92:0/4262671705 conn(0x7fda84073bc0 msgr2=0x7fda8414ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.737+0000 7fda8c2a0640 1 -- 172.21.15.92:0/4262671705 shutdown_connections 2024-04-22T07:04:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.737+0000 7fda8c2a0640 1 -- 172.21.15.92:0/4262671705 wait complete. 2024-04-22T07:04:08.746 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-22T07:04:07.933627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218928,"kb_used_data":2760,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755792,"statfs":{"total":773094113280,"available":772869931008,"internally_reserved":0,"allocated":2826240,"data_stored":1748208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.157756"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862171+0000","last_change":"2024-04-22T07:04:03.796924+0000","last_active":"2024-04-22T07:04:04.862171+0000","last_peered":"2024-04-22T07:04:04.862171+0000","last_clean":"2024-04-22T07:04:04.862171+0000","last_became_active":"2024-04-22T07:04:03.796652+0000","last_became_peered":"2024-04-22T07:04:03.796652+0000","last_unstale":"2024-04-22T07:04:04.862171+0000","last_undegraded":"2024-04-22T07:04:04.862171+0000","last_fullsized":"2024-04-22T07:04:04.862171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:01.867906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788117+0000","last_change":"2024-04-22T07:04:03.793652+0000","last_active":"2024-04-22T07:04:04.788117+0000","last_peered":"2024-04-22T07:04:04.788117+0000","last_clean":"2024-04-22T07:04:04.788117+0000","last_became_active":"2024-04-22T07:04:03.793397+0000","last_became_peered":"2024-04-22T07:04:03.793397+0000","last_unstale":"2024-04-22T07:04:04.788117+0000","last_undegraded":"2024-04-22T07:04:04.788117+0000","last_fullsized":"2024-04-22T07:04:04.788117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859908+0000","last_change":"2024-04-22T07:04:03.788534+0000","last_active":"2024-04-22T07:04:04.859908+0000","last_peered":"2024-04-22T07:04:04.859908+0000","last_clean":"2024-04-22T07:04:04.859908+0000","last_became_active":"2024-04-22T07:04:03.787747+0000","last_became_peered":"2024-04-22T07:04:03.787747+0000","last_unstale":"2024-04-22T07:04:04.859908+0000","last_undegraded":"2024-04-22T07:04:04.859908+0000","last_fullsized":"2024-04-22T07:04:04.859908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861337+0000","last_change":"2024-04-22T07:04:03.800150+0000","last_active":"2024-04-22T07:04:04.861337+0000","last_peered":"2024-04-22T07:04:04.861337+0000","last_clean":"2024-04-22T07:04:04.861337+0000","last_became_active":"2024-04-22T07:04:03.799887+0000","last_became_peered":"2024-04-22T07:04:03.799887+0000","last_unstale":"2024-04-22T07:04:04.861337+0000","last_undegraded":"2024-04-22T07:04:04.861337+0000","last_fullsized":"2024-04-22T07:04:04.861337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861727+0000","last_change":"2024-04-22T07:04:03.798482+0000","last_active":"2024-04-22T07:04:04.861727+0000","last_peered":"2024-04-22T07:04:04.861727+0000","last_clean":"2024-04-22T07:04:04.861727+0000","last_became_active":"2024-04-22T07:04:03.798205+0000","last_became_peered":"2024-04-22T07:04:03.798205+0000","last_unstale":"2024-04-22T07:04:04.861727+0000","last_undegraded":"2024-04-22T07:04:04.861727+0000","last_fullsized":"2024-04-22T07:04:04.861727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861044+0000","last_change":"2024-04-22T07:04:03.797373+0000","last_active":"2024-04-22T07:04:04.861044+0000","last_peered":"2024-04-22T07:04:04.861044+0000","last_clean":"2024-04-22T07:04:04.861044+0000","last_became_active":"2024-04-22T07:04:03.797090+0000","last_became_peered":"2024-04-22T07:04:03.797090+0000","last_unstale":"2024-04-22T07:04:04.861044+0000","last_undegraded":"2024-04-22T07:04:04.861044+0000","last_fullsized":"2024-04-22T07:04:04.861044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861437+0000","last_change":"2024-04-22T07:04:03.793725+0000","last_active":"2024-04-22T07:04:04.861437+0000","last_peered":"2024-04-22T07:04:04.861437+0000","last_clean":"2024-04-22T07:04:04.861437+0000","last_became_active":"2024-04-22T07:04:03.793418+0000","last_became_peered":"2024-04-22T07:04:03.793418+0000","last_unstale":"2024-04-22T07:04:04.861437+0000","last_undegraded":"2024-04-22T07:04:04.861437+0000","last_fullsized":"2024-04-22T07:04:04.861437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859500+0000","last_change":"2024-04-22T07:04:03.795377+0000","last_active":"2024-04-22T07:04:04.859500+0000","last_peered":"2024-04-22T07:04:04.859500+0000","last_clean":"2024-04-22T07:04:04.859500+0000","last_became_active":"2024-04-22T07:04:03.795106+0000","last_became_peered":"2024-04-22T07:04:03.795106+0000","last_unstale":"2024-04-22T07:04:04.859500+0000","last_undegraded":"2024-04-22T07:04:04.859500+0000","last_fullsized":"2024-04-22T07:04:04.859500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862209+0000","last_change":"2024-04-22T07:04:03.797692+0000","last_active":"2024-04-22T07:04:04.862209+0000","last_peered":"2024-04-22T07:04:04.862209+0000","last_clean":"2024-04-22T07:04:04.862209+0000","last_became_active":"2024-04-22T07:04:03.797396+0000","last_became_peered":"2024-04-22T07:04:03.797396+0000","last_unstale":"2024-04-22T07:04:04.862209+0000","last_undegraded":"2024-04-22T07:04:04.862209+0000","last_fullsized":"2024-04-22T07:04:04.862209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:39.068857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860742+0000","last_change":"2024-04-22T07:04:03.797228+0000","last_active":"2024-04-22T07:04:04.860742+0000","last_peered":"2024-04-22T07:04:04.860742+0000","last_clean":"2024-04-22T07:04:04.860742+0000","last_became_active":"2024-04-22T07:04:03.796614+0000","last_became_peered":"2024-04-22T07:04:03.796614+0000","last_unstale":"2024-04-22T07:04:04.860742+0000","last_undegraded":"2024-04-22T07:04:04.860742+0000","last_fullsized":"2024-04-22T07:04:04.860742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860497+0000","last_change":"2024-04-22T07:04:03.799434+0000","last_active":"2024-04-22T07:04:04.860497+0000","last_peered":"2024-04-22T07:04:04.860497+0000","last_clean":"2024-04-22T07:04:04.860497+0000","last_became_active":"2024-04-22T07:04:03.799173+0000","last_became_peered":"2024-04-22T07:04:03.799173+0000","last_unstale":"2024-04-22T07:04:04.860497+0000","last_undegraded":"2024-04-22T07:04:04.860497+0000","last_fullsized":"2024-04-22T07:04:04.860497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790488+0000","last_change":"2024-04-22T07:04:03.800035+0000","last_active":"2024-04-22T07:04:04.790488+0000","last_peered":"2024-04-22T07:04:04.790488+0000","last_clean":"2024-04-22T07:04:04.790488+0000","last_became_active":"2024-04-22T07:04:03.799646+0000","last_became_peered":"2024-04-22T07:04:03.799646+0000","last_unstale":"2024-04-22T07:04:04.790488+0000","last_undegraded":"2024-04-22T07:04:04.790488+0000","last_fullsized":"2024-04-22T07:04:04.790488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:17.287068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860651+0000","last_change":"2024-04-22T07:04:03.797324+0000","last_active":"2024-04-22T07:04:04.860651+0000","last_peered":"2024-04-22T07:04:04.860651+0000","last_clean":"2024-04-22T07:04:04.860651+0000","last_became_active":"2024-04-22T07:04:03.797039+0000","last_became_peered":"2024-04-22T07:04:03.797039+0000","last_unstale":"2024-04-22T07:04:04.860651+0000","last_undegraded":"2024-04-22T07:04:04.860651+0000","last_fullsized":"2024-04-22T07:04:04.860651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859720+0000","last_change":"2024-04-22T07:04:03.790732+0000","last_active":"2024-04-22T07:04:04.859720+0000","last_peered":"2024-04-22T07:04:04.859720+0000","last_clean":"2024-04-22T07:04:04.859720+0000","last_became_active":"2024-04-22T07:04:03.788340+0000","last_became_peered":"2024-04-22T07:04:03.788340+0000","last_unstale":"2024-04-22T07:04:04.859720+0000","last_undegraded":"2024-04-22T07:04:04.859720+0000","last_fullsized":"2024-04-22T07:04:04.859720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787860+0000","last_change":"2024-04-22T07:04:03.799368+0000","last_active":"2024-04-22T07:04:04.787860+0000","last_peered":"2024-04-22T07:04:04.787860+0000","last_clean":"2024-04-22T07:04:04.787860+0000","last_became_active":"2024-04-22T07:04:03.799067+0000","last_became_peered":"2024-04-22T07:04:03.799067+0000","last_unstale":"2024-04-22T07:04:04.787860+0000","last_undegraded":"2024-04-22T07:04:04.787860+0000","last_fullsized":"2024-04-22T07:04:04.787860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860988+0000","last_change":"2024-04-22T07:04:03.802947+0000","last_active":"2024-04-22T07:04:04.860988+0000","last_peered":"2024-04-22T07:04:04.860988+0000","last_clean":"2024-04-22T07:04:04.860988+0000","last_became_active":"2024-04-22T07:04:03.802493+0000","last_became_peered":"2024-04-22T07:04:03.802493+0000","last_unstale":"2024-04-22T07:04:04.860988+0000","last_undegraded":"2024-04-22T07:04:04.860988+0000","last_fullsized":"2024-04-22T07:04:04.860988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860714+0000","last_change":"2024-04-22T07:04:03.795581+0000","last_active":"2024-04-22T07:04:04.860714+0000","last_peered":"2024-04-22T07:04:04.860714+0000","last_clean":"2024-04-22T07:04:04.860714+0000","last_became_active":"2024-04-22T07:04:03.795292+0000","last_became_peered":"2024-04-22T07:04:03.795292+0000","last_unstale":"2024-04-22T07:04:04.860714+0000","last_undegraded":"2024-04-22T07:04:04.860714+0000","last_fullsized":"2024-04-22T07:04:04.860714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859993+0000","last_change":"2024-04-22T07:04:03.796132+0000","last_active":"2024-04-22T07:04:04.859993+0000","last_peered":"2024-04-22T07:04:04.859993+0000","last_clean":"2024-04-22T07:04:04.859993+0000","last_became_active":"2024-04-22T07:04:03.795069+0000","last_became_peered":"2024-04-22T07:04:03.795069+0000","last_unstale":"2024-04-22T07:04:04.859993+0000","last_undegraded":"2024-04-22T07:04:04.859993+0000","last_fullsized":"2024-04-22T07:04:04.859993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:08.934970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862125+0000","last_change":"2024-04-22T07:04:03.794552+0000","last_active":"2024-04-22T07:04:04.862125+0000","last_peered":"2024-04-22T07:04:04.862125+0000","last_clean":"2024-04-22T07:04:04.862125+0000","last_became_active":"2024-04-22T07:04:03.794046+0000","last_became_peered":"2024-04-22T07:04:03.794046+0000","last_unstale":"2024-04-22T07:04:04.862125+0000","last_undegraded":"2024-04-22T07:04:04.862125+0000","last_fullsized":"2024-04-22T07:04:04.862125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860703+0000","last_change":"2024-04-22T07:04:03.798686+0000","last_active":"2024-04-22T07:04:04.860703+0000","last_peered":"2024-04-22T07:04:04.860703+0000","last_clean":"2024-04-22T07:04:04.860703+0000","last_became_active":"2024-04-22T07:04:03.798104+0000","last_became_peered":"2024-04-22T07:04:03.798104+0000","last_unstale":"2024-04-22T07:04:04.860703+0000","last_undegraded":"2024-04-22T07:04:04.860703+0000","last_fullsized":"2024-04-22T07:04:04.860703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860494+0000","last_change":"2024-04-22T07:04:03.794215+0000","last_active":"2024-04-22T07:04:04.860494+0000","last_peered":"2024-04-22T07:04:04.860494+0000","last_clean":"2024-04-22T07:04:04.860494+0000","last_became_active":"2024-04-22T07:04:03.793990+0000","last_became_peered":"2024-04-22T07:04:03.793990+0000","last_unstale":"2024-04-22T07:04:04.860494+0000","last_undegraded":"2024-04-22T07:04:04.860494+0000","last_fullsized":"2024-04-22T07:04:04.860494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861162+0000","last_change":"2024-04-22T07:04:03.798539+0000","last_active":"2024-04-22T07:04:04.861162+0000","last_peered":"2024-04-22T07:04:04.861162+0000","last_clean":"2024-04-22T07:04:04.861162+0000","last_became_active":"2024-04-22T07:04:03.797713+0000","last_became_peered":"2024-04-22T07:04:03.797713+0000","last_unstale":"2024-04-22T07:04:04.861162+0000","last_undegraded":"2024-04-22T07:04:04.861162+0000","last_fullsized":"2024-04-22T07:04:04.861162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859945+0000","last_change":"2024-04-22T07:04:03.798610+0000","last_active":"2024-04-22T07:04:04.859945+0000","last_peered":"2024-04-22T07:04:04.859945+0000","last_clean":"2024-04-22T07:04:04.859945+0000","last_became_active":"2024-04-22T07:04:03.797981+0000","last_became_peered":"2024-04-22T07:04:03.797981+0000","last_unstale":"2024-04-22T07:04:04.859945+0000","last_undegraded":"2024-04-22T07:04:04.859945+0000","last_fullsized":"2024-04-22T07:04:04.859945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861507+0000","last_change":"2024-04-22T07:04:03.802742+0000","last_active":"2024-04-22T07:04:04.861507+0000","last_peered":"2024-04-22T07:04:04.861507+0000","last_clean":"2024-04-22T07:04:04.861507+0000","last_became_active":"2024-04-22T07:04:03.802127+0000","last_became_peered":"2024-04-22T07:04:03.802127+0000","last_unstale":"2024-04-22T07:04:04.861507+0000","last_undegraded":"2024-04-22T07:04:04.861507+0000","last_fullsized":"2024-04-22T07:04:04.861507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860138+0000","last_change":"2024-04-22T07:04:03.798789+0000","last_active":"2024-04-22T07:04:04.860138+0000","last_peered":"2024-04-22T07:04:04.860138+0000","last_clean":"2024-04-22T07:04:04.860138+0000","last_became_active":"2024-04-22T07:04:03.798505+0000","last_became_peered":"2024-04-22T07:04:03.798505+0000","last_unstale":"2024-04-22T07:04:04.860138+0000","last_undegraded":"2024-04-22T07:04:04.860138+0000","last_fullsized":"2024-04-22T07:04:04.860138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860919+0000","last_change":"2024-04-22T07:04:03.794111+0000","last_active":"2024-04-22T07:04:04.860919+0000","last_peered":"2024-04-22T07:04:04.860919+0000","last_clean":"2024-04-22T07:04:04.860919+0000","last_became_active":"2024-04-22T07:04:03.793682+0000","last_became_peered":"2024-04-22T07:04:03.793682+0000","last_unstale":"2024-04-22T07:04:04.860919+0000","last_undegraded":"2024-04-22T07:04:04.860919+0000","last_fullsized":"2024-04-22T07:04:04.860919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788083+0000","last_change":"2024-04-22T07:04:03.790167+0000","last_active":"2024-04-22T07:04:04.788083+0000","last_peered":"2024-04-22T07:04:04.788083+0000","last_clean":"2024-04-22T07:04:04.788083+0000","last_became_active":"2024-04-22T07:04:03.789873+0000","last_became_peered":"2024-04-22T07:04:03.789873+0000","last_unstale":"2024-04-22T07:04:04.788083+0000","last_undegraded":"2024-04-22T07:04:04.788083+0000","last_fullsized":"2024-04-22T07:04:04.788083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859954+0000","last_change":"2024-04-22T07:04:03.798713+0000","last_active":"2024-04-22T07:04:04.859954+0000","last_peered":"2024-04-22T07:04:04.859954+0000","last_clean":"2024-04-22T07:04:04.859954+0000","last_became_active":"2024-04-22T07:04:03.798344+0000","last_became_peered":"2024-04-22T07:04:03.798344+0000","last_unstale":"2024-04-22T07:04:04.859954+0000","last_undegraded":"2024-04-22T07:04:04.859954+0000","last_fullsized":"2024-04-22T07:04:04.859954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860634+0000","last_change":"2024-04-22T07:04:03.795452+0000","last_active":"2024-04-22T07:04:04.860634+0000","last_peered":"2024-04-22T07:04:04.860634+0000","last_clean":"2024-04-22T07:04:04.860634+0000","last_became_active":"2024-04-22T07:04:03.794970+0000","last_became_peered":"2024-04-22T07:04:03.794970+0000","last_unstale":"2024-04-22T07:04:04.860634+0000","last_undegraded":"2024-04-22T07:04:04.860634+0000","last_fullsized":"2024-04-22T07:04:04.860634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861203+0000","last_change":"2024-04-22T07:04:03.799581+0000","last_active":"2024-04-22T07:04:04.861203+0000","last_peered":"2024-04-22T07:04:04.861203+0000","last_clean":"2024-04-22T07:04:04.861203+0000","last_became_active":"2024-04-22T07:04:03.799328+0000","last_became_peered":"2024-04-22T07:04:03.799328+0000","last_unstale":"2024-04-22T07:04:04.861203+0000","last_undegraded":"2024-04-22T07:04:04.861203+0000","last_fullsized":"2024-04-22T07:04:04.861203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859944+0000","last_change":"2024-04-22T07:04:03.796830+0000","last_active":"2024-04-22T07:04:04.859944+0000","last_peered":"2024-04-22T07:04:04.859944+0000","last_clean":"2024-04-22T07:04:04.859944+0000","last_became_active":"2024-04-22T07:04:03.796551+0000","last_became_peered":"2024-04-22T07:04:03.796551+0000","last_unstale":"2024-04-22T07:04:04.859944+0000","last_undegraded":"2024-04-22T07:04:04.859944+0000","last_fullsized":"2024-04-22T07:04:04.859944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860101+0000","last_change":"2024-04-22T07:04:03.795320+0000","last_active":"2024-04-22T07:04:04.860101+0000","last_peered":"2024-04-22T07:04:04.860101+0000","last_clean":"2024-04-22T07:04:04.860101+0000","last_became_active":"2024-04-22T07:04:03.795025+0000","last_became_peered":"2024-04-22T07:04:03.795025+0000","last_unstale":"2024-04-22T07:04:04.860101+0000","last_undegraded":"2024-04-22T07:04:04.860101+0000","last_fullsized":"2024-04-22T07:04:04.860101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790119+0000","last_change":"2024-04-22T07:04:03.788859+0000","last_active":"2024-04-22T07:04:04.790119+0000","last_peered":"2024-04-22T07:04:04.790119+0000","last_clean":"2024-04-22T07:04:04.790119+0000","last_became_active":"2024-04-22T07:04:03.788573+0000","last_became_peered":"2024-04-22T07:04:03.788573+0000","last_unstale":"2024-04-22T07:04:04.790119+0000","last_undegraded":"2024-04-22T07:04:04.790119+0000","last_fullsized":"2024-04-22T07:04:04.790119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:36.716689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860694+0000","last_change":"2024-04-22T07:04:02.795724+0000","last_active":"2024-04-22T07:04:04.860694+0000","last_peered":"2024-04-22T07:04:04.860694+0000","last_clean":"2024-04-22T07:04:04.860694+0000","last_became_active":"2024-04-22T07:04:02.795371+0000","last_became_peered":"2024-04-22T07:04:02.795371+0000","last_unstale":"2024-04-22T07:04:04.860694+0000","last_undegraded":"2024-04-22T07:04:04.860694+0000","last_fullsized":"2024-04-22T07:04:04.860694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860689+0000","last_change":"2024-04-22T07:04:03.787030+0000","last_active":"2024-04-22T07:04:04.860689+0000","last_peered":"2024-04-22T07:04:04.860689+0000","last_clean":"2024-04-22T07:04:04.860689+0000","last_became_active":"2024-04-22T07:04:03.786357+0000","last_became_peered":"2024-04-22T07:04:03.786357+0000","last_unstale":"2024-04-22T07:04:04.860689+0000","last_undegraded":"2024-04-22T07:04:04.860689+0000","last_fullsized":"2024-04-22T07:04:04.860689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860316+0000","last_change":"2024-04-22T07:04:02.791429+0000","last_active":"2024-04-22T07:04:04.860316+0000","last_peered":"2024-04-22T07:04:04.860316+0000","last_clean":"2024-04-22T07:04:04.860316+0000","last_became_active":"2024-04-22T07:04:02.791114+0000","last_became_peered":"2024-04-22T07:04:02.791114+0000","last_unstale":"2024-04-22T07:04:04.860316+0000","last_undegraded":"2024-04-22T07:04:04.860316+0000","last_fullsized":"2024-04-22T07:04:04.860316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860040+0000","last_change":"2024-04-22T07:04:02.789531+0000","last_active":"2024-04-22T07:04:04.860040+0000","last_peered":"2024-04-22T07:04:04.860040+0000","last_clean":"2024-04-22T07:04:04.860040+0000","last_became_active":"2024-04-22T07:04:02.789248+0000","last_became_peered":"2024-04-22T07:04:02.789248+0000","last_unstale":"2024-04-22T07:04:04.860040+0000","last_undegraded":"2024-04-22T07:04:04.860040+0000","last_fullsized":"2024-04-22T07:04:04.860040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:06:43.031782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859624+0000","last_change":"2024-04-22T07:04:03.790142+0000","last_active":"2024-04-22T07:04:04.859624+0000","last_peered":"2024-04-22T07:04:04.859624+0000","last_clean":"2024-04-22T07:04:04.859624+0000","last_became_active":"2024-04-22T07:04:03.789869+0000","last_became_peered":"2024-04-22T07:04:03.789869+0000","last_unstale":"2024-04-22T07:04:04.859624+0000","last_undegraded":"2024-04-22T07:04:04.859624+0000","last_fullsized":"2024-04-22T07:04:04.859624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861050+0000","last_change":"2024-04-22T07:04:02.790537+0000","last_active":"2024-04-22T07:04:04.861050+0000","last_peered":"2024-04-22T07:04:04.861050+0000","last_clean":"2024-04-22T07:04:04.861050+0000","last_became_active":"2024-04-22T07:04:02.789072+0000","last_became_peered":"2024-04-22T07:04:02.789072+0000","last_unstale":"2024-04-22T07:04:04.861050+0000","last_undegraded":"2024-04-22T07:04:04.861050+0000","last_fullsized":"2024-04-22T07:04:04.861050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787835+0000","last_change":"2024-04-22T07:04:03.791239+0000","last_active":"2024-04-22T07:04:04.787835+0000","last_peered":"2024-04-22T07:04:04.787835+0000","last_clean":"2024-04-22T07:04:04.787835+0000","last_became_active":"2024-04-22T07:04:03.790515+0000","last_became_peered":"2024-04-22T07:04:03.790515+0000","last_unstale":"2024-04-22T07:04:04.787835+0000","last_undegraded":"2024-04-22T07:04:04.787835+0000","last_fullsized":"2024-04-22T07:04:04.787835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860666+0000","last_change":"2024-04-22T07:04:02.793582+0000","last_active":"2024-04-22T07:04:04.860666+0000","last_peered":"2024-04-22T07:04:04.860666+0000","last_clean":"2024-04-22T07:04:04.860666+0000","last_became_active":"2024-04-22T07:04:02.793281+0000","last_became_peered":"2024-04-22T07:04:02.793281+0000","last_unstale":"2024-04-22T07:04:04.860666+0000","last_undegraded":"2024-04-22T07:04:04.860666+0000","last_fullsized":"2024-04-22T07:04:04.860666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860510+0000","last_change":"2024-04-22T07:04:03.793994+0000","last_active":"2024-04-22T07:04:04.860510+0000","last_peered":"2024-04-22T07:04:04.860510+0000","last_clean":"2024-04-22T07:04:04.860510+0000","last_became_active":"2024-04-22T07:04:03.793734+0000","last_became_peered":"2024-04-22T07:04:03.793734+0000","last_unstale":"2024-04-22T07:04:04.860510+0000","last_undegraded":"2024-04-22T07:04:04.860510+0000","last_fullsized":"2024-04-22T07:04:04.860510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860572+0000","last_change":"2024-04-22T07:04:02.791945+0000","last_active":"2024-04-22T07:04:04.860572+0000","last_peered":"2024-04-22T07:04:04.860572+0000","last_clean":"2024-04-22T07:04:04.860572+0000","last_became_active":"2024-04-22T07:04:02.791654+0000","last_became_peered":"2024-04-22T07:04:02.791654+0000","last_unstale":"2024-04-22T07:04:04.860572+0000","last_undegraded":"2024-04-22T07:04:04.860572+0000","last_fullsized":"2024-04-22T07:04:04.860572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860230+0000","last_change":"2024-04-22T07:04:03.797391+0000","last_active":"2024-04-22T07:04:04.860230+0000","last_peered":"2024-04-22T07:04:04.860230+0000","last_clean":"2024-04-22T07:04:04.860230+0000","last_became_active":"2024-04-22T07:04:03.797123+0000","last_became_peered":"2024-04-22T07:04:03.797123+0000","last_unstale":"2024-04-22T07:04:04.860230+0000","last_undegraded":"2024-04-22T07:04:04.860230+0000","last_fullsized":"2024-04-22T07:04:04.860230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788045+0000","last_change":"2024-04-22T07:04:02.793753+0000","last_active":"2024-04-22T07:04:04.788045+0000","last_peered":"2024-04-22T07:04:04.788045+0000","last_clean":"2024-04-22T07:04:04.788045+0000","last_became_active":"2024-04-22T07:04:02.793436+0000","last_became_peered":"2024-04-22T07:04:02.793436+0000","last_unstale":"2024-04-22T07:04:04.788045+0000","last_undegraded":"2024-04-22T07:04:04.788045+0000","last_fullsized":"2024-04-22T07:04:04.788045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790527+0000","last_change":"2024-04-22T07:04:03.800665+0000","last_active":"2024-04-22T07:04:04.790527+0000","last_peered":"2024-04-22T07:04:04.790527+0000","last_clean":"2024-04-22T07:04:04.790527+0000","last_became_active":"2024-04-22T07:04:03.800227+0000","last_became_peered":"2024-04-22T07:04:03.800227+0000","last_unstale":"2024-04-22T07:04:04.790527+0000","last_undegraded":"2024-04-22T07:04:04.790527+0000","last_fullsized":"2024-04-22T07:04:04.790527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860760+0000","last_change":"2024-04-22T07:04:02.791309+0000","last_active":"2024-04-22T07:04:04.860760+0000","last_peered":"2024-04-22T07:04:04.860760+0000","last_clean":"2024-04-22T07:04:04.860760+0000","last_became_active":"2024-04-22T07:04:02.791000+0000","last_became_peered":"2024-04-22T07:04:02.791000+0000","last_unstale":"2024-04-22T07:04:04.860760+0000","last_undegraded":"2024-04-22T07:04:04.860760+0000","last_fullsized":"2024-04-22T07:04:04.860760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861543+0000","last_change":"2024-04-22T07:04:03.794316+0000","last_active":"2024-04-22T07:04:04.861543+0000","last_peered":"2024-04-22T07:04:04.861543+0000","last_clean":"2024-04-22T07:04:04.861543+0000","last_became_active":"2024-04-22T07:04:03.793646+0000","last_became_peered":"2024-04-22T07:04:03.793646+0000","last_unstale":"2024-04-22T07:04:04.861543+0000","last_undegraded":"2024-04-22T07:04:04.861543+0000","last_fullsized":"2024-04-22T07:04:04.861543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860673+0000","last_change":"2024-04-22T07:04:02.791024+0000","last_active":"2024-04-22T07:04:04.860673+0000","last_peered":"2024-04-22T07:04:04.860673+0000","last_clean":"2024-04-22T07:04:04.860673+0000","last_became_active":"2024-04-22T07:04:02.789160+0000","last_became_peered":"2024-04-22T07:04:02.789160+0000","last_unstale":"2024-04-22T07:04:04.860673+0000","last_undegraded":"2024-04-22T07:04:04.860673+0000","last_fullsized":"2024-04-22T07:04:04.860673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859792+0000","last_change":"2024-04-22T07:04:03.798436+0000","last_active":"2024-04-22T07:04:04.859792+0000","last_peered":"2024-04-22T07:04:04.859792+0000","last_clean":"2024-04-22T07:04:04.859792+0000","last_became_active":"2024-04-22T07:04:03.797946+0000","last_became_peered":"2024-04-22T07:04:03.797946+0000","last_unstale":"2024-04-22T07:04:04.859792+0000","last_undegraded":"2024-04-22T07:04:04.859792+0000","last_fullsized":"2024-04-22T07:04:04.859792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861234+0000","last_change":"2024-04-22T07:04:02.796551+0000","last_active":"2024-04-22T07:04:04.861234+0000","last_peered":"2024-04-22T07:04:04.861234+0000","last_clean":"2024-04-22T07:04:04.861234+0000","last_became_active":"2024-04-22T07:04:02.796332+0000","last_became_peered":"2024-04-22T07:04:02.796332+0000","last_unstale":"2024-04-22T07:04:04.861234+0000","last_undegraded":"2024-04-22T07:04:04.861234+0000","last_fullsized":"2024-04-22T07:04:04.861234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:15.854295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861758+0000","last_change":"2024-04-22T07:04:03.804622+0000","last_active":"2024-04-22T07:04:04.861758+0000","last_peered":"2024-04-22T07:04:04.861758+0000","last_clean":"2024-04-22T07:04:04.861758+0000","last_became_active":"2024-04-22T07:04:03.804324+0000","last_became_peered":"2024-04-22T07:04:03.804324+0000","last_unstale":"2024-04-22T07:04:04.861758+0000","last_undegraded":"2024-04-22T07:04:04.861758+0000","last_fullsized":"2024-04-22T07:04:04.861758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860167+0000","last_change":"2024-04-22T07:04:02.787085+0000","last_active":"2024-04-22T07:04:04.860167+0000","last_peered":"2024-04-22T07:04:04.860167+0000","last_clean":"2024-04-22T07:04:04.860167+0000","last_became_active":"2024-04-22T07:04:02.786777+0000","last_became_peered":"2024-04-22T07:04:02.786777+0000","last_unstale":"2024-04-22T07:04:04.860167+0000","last_undegraded":"2024-04-22T07:04:04.860167+0000","last_fullsized":"2024-04-22T07:04:04.860167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860535+0000","last_change":"2024-04-22T07:04:03.796356+0000","last_active":"2024-04-22T07:04:04.860535+0000","last_peered":"2024-04-22T07:04:04.860535+0000","last_clean":"2024-04-22T07:04:04.860535+0000","last_became_active":"2024-04-22T07:04:03.796030+0000","last_became_peered":"2024-04-22T07:04:03.796030+0000","last_unstale":"2024-04-22T07:04:04.860535+0000","last_undegraded":"2024-04-22T07:04:04.860535+0000","last_fullsized":"2024-04-22T07:04:04.860535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860233+0000","last_change":"2024-04-22T07:04:02.787425+0000","last_active":"2024-04-22T07:04:04.860233+0000","last_peered":"2024-04-22T07:04:04.860233+0000","last_clean":"2024-04-22T07:04:04.860233+0000","last_became_active":"2024-04-22T07:04:02.787146+0000","last_became_peered":"2024-04-22T07:04:02.787146+0000","last_unstale":"2024-04-22T07:04:04.860233+0000","last_undegraded":"2024-04-22T07:04:04.860233+0000","last_fullsized":"2024-04-22T07:04:04.860233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861631+0000","last_change":"2024-04-22T07:04:03.802924+0000","last_active":"2024-04-22T07:04:04.861631+0000","last_peered":"2024-04-22T07:04:04.861631+0000","last_clean":"2024-04-22T07:04:04.861631+0000","last_became_active":"2024-04-22T07:04:03.802644+0000","last_became_peered":"2024-04-22T07:04:03.802644+0000","last_unstale":"2024-04-22T07:04:04.861631+0000","last_undegraded":"2024-04-22T07:04:04.861631+0000","last_fullsized":"2024-04-22T07:04:04.861631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790380+0000","last_change":"2024-04-22T07:04:02.794481+0000","last_active":"2024-04-22T07:04:04.790380+0000","last_peered":"2024-04-22T07:04:04.790380+0000","last_clean":"2024-04-22T07:04:04.790380+0000","last_became_active":"2024-04-22T07:04:02.793982+0000","last_became_peered":"2024-04-22T07:04:02.793982+0000","last_unstale":"2024-04-22T07:04:04.790380+0000","last_undegraded":"2024-04-22T07:04:04.790380+0000","last_fullsized":"2024-04-22T07:04:04.790380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:58.906416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862139+0000","last_change":"2024-04-22T07:04:03.802724+0000","last_active":"2024-04-22T07:04:04.862139+0000","last_peered":"2024-04-22T07:04:04.862139+0000","last_clean":"2024-04-22T07:04:04.862139+0000","last_became_active":"2024-04-22T07:04:03.802434+0000","last_became_peered":"2024-04-22T07:04:03.802434+0000","last_unstale":"2024-04-22T07:04:04.862139+0000","last_undegraded":"2024-04-22T07:04:04.862139+0000","last_fullsized":"2024-04-22T07:04:04.862139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860035+0000","last_change":"2024-04-22T07:04:02.791375+0000","last_active":"2024-04-22T07:04:04.860035+0000","last_peered":"2024-04-22T07:04:04.860035+0000","last_clean":"2024-04-22T07:04:04.860035+0000","last_became_active":"2024-04-22T07:04:02.791095+0000","last_became_peered":"2024-04-22T07:04:02.791095+0000","last_unstale":"2024-04-22T07:04:04.860035+0000","last_undegraded":"2024-04-22T07:04:04.860035+0000","last_fullsized":"2024-04-22T07:04:04.860035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862265+0000","last_change":"2024-04-22T07:04:03.802028+0000","last_active":"2024-04-22T07:04:04.862265+0000","last_peered":"2024-04-22T07:04:04.862265+0000","last_clean":"2024-04-22T07:04:04.862265+0000","last_became_active":"2024-04-22T07:04:03.801715+0000","last_became_peered":"2024-04-22T07:04:03.801715+0000","last_unstale":"2024-04-22T07:04:04.862265+0000","last_undegraded":"2024-04-22T07:04:04.862265+0000","last_fullsized":"2024-04-22T07:04:04.862265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:29.815068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788119+0000","last_change":"2024-04-22T07:04:02.793270+0000","last_active":"2024-04-22T07:04:04.788119+0000","last_peered":"2024-04-22T07:04:04.788119+0000","last_clean":"2024-04-22T07:04:04.788119+0000","last_became_active":"2024-04-22T07:04:02.793009+0000","last_became_peered":"2024-04-22T07:04:02.793009+0000","last_unstale":"2024-04-22T07:04:04.788119+0000","last_undegraded":"2024-04-22T07:04:04.788119+0000","last_fullsized":"2024-04-22T07:04:04.788119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861459+0000","last_change":"2024-04-22T07:04:03.800175+0000","last_active":"2024-04-22T07:04:04.861459+0000","last_peered":"2024-04-22T07:04:04.861459+0000","last_clean":"2024-04-22T07:04:04.861459+0000","last_became_active":"2024-04-22T07:04:03.799902+0000","last_became_peered":"2024-04-22T07:04:03.799902+0000","last_unstale":"2024-04-22T07:04:04.861459+0000","last_undegraded":"2024-04-22T07:04:04.861459+0000","last_fullsized":"2024-04-22T07:04:04.861459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859806+0000","last_change":"2024-04-22T07:04:02.793643+0000","last_active":"2024-04-22T07:04:04.859806+0000","last_peered":"2024-04-22T07:04:04.859806+0000","last_clean":"2024-04-22T07:04:04.859806+0000","last_became_active":"2024-04-22T07:04:02.793333+0000","last_became_peered":"2024-04-22T07:04:02.793333+0000","last_unstale":"2024-04-22T07:04:04.859806+0000","last_undegraded":"2024-04-22T07:04:04.859806+0000","last_fullsized":"2024-04-22T07:04:04.859806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862019+0000","last_change":"2024-04-22T07:04:03.801821+0000","last_active":"2024-04-22T07:04:04.862019+0000","last_peered":"2024-04-22T07:04:04.862019+0000","last_clean":"2024-04-22T07:04:04.862019+0000","last_became_active":"2024-04-22T07:04:03.801510+0000","last_became_peered":"2024-04-22T07:04:03.801510+0000","last_unstale":"2024-04-22T07:04:04.862019+0000","last_undegraded":"2024-04-22T07:04:04.862019+0000","last_fullsized":"2024-04-22T07:04:04.862019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861241+0000","last_change":"2024-04-22T07:03:59.215285+0000","last_active":"2024-04-22T07:04:04.861241+0000","last_peered":"2024-04-22T07:04:04.861241+0000","last_clean":"2024-04-22T07:04:04.861241+0000","last_became_active":"2024-04-22T07:03:59.214275+0000","last_became_peered":"2024-04-22T07:03:59.214275+0000","last_unstale":"2024-04-22T07:04:04.861241+0000","last_undegraded":"2024-04-22T07:04:04.861241+0000","last_fullsized":"2024-04-22T07:04:04.861241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_clean_scrub_stamp":"2024-04-22T07:03:57.959761+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:55.105073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860226+0000","last_change":"2024-04-22T07:04:02.793384+0000","last_active":"2024-04-22T07:04:04.860226+0000","last_peered":"2024-04-22T07:04:04.860226+0000","last_clean":"2024-04-22T07:04:04.860226+0000","last_became_active":"2024-04-22T07:04:02.793077+0000","last_became_peered":"2024-04-22T07:04:02.793077+0000","last_unstale":"2024-04-22T07:04:04.860226+0000","last_undegraded":"2024-04-22T07:04:04.860226+0000","last_fullsized":"2024-04-22T07:04:04.860226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860779+0000","last_change":"2024-04-22T07:04:03.798134+0000","last_active":"2024-04-22T07:04:04.860779+0000","last_peered":"2024-04-22T07:04:04.860779+0000","last_clean":"2024-04-22T07:04:04.860779+0000","last_became_active":"2024-04-22T07:04:03.797848+0000","last_became_peered":"2024-04-22T07:04:03.797848+0000","last_unstale":"2024-04-22T07:04:04.860779+0000","last_undegraded":"2024-04-22T07:04:04.860779+0000","last_fullsized":"2024-04-22T07:04:04.860779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859966+0000","last_change":"2024-04-22T07:04:02.789591+0000","last_active":"2024-04-22T07:04:04.859966+0000","last_peered":"2024-04-22T07:04:04.859966+0000","last_clean":"2024-04-22T07:04:04.859966+0000","last_became_active":"2024-04-22T07:04:02.788785+0000","last_became_peered":"2024-04-22T07:04:02.788785+0000","last_unstale":"2024-04-22T07:04:04.859966+0000","last_undegraded":"2024-04-22T07:04:04.859966+0000","last_fullsized":"2024-04-22T07:04:04.859966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859611+0000","last_change":"2024-04-22T07:04:03.789757+0000","last_active":"2024-04-22T07:04:04.859611+0000","last_peered":"2024-04-22T07:04:04.859611+0000","last_clean":"2024-04-22T07:04:04.859611+0000","last_became_active":"2024-04-22T07:04:03.789463+0000","last_became_peered":"2024-04-22T07:04:03.789463+0000","last_unstale":"2024-04-22T07:04:04.859611+0000","last_undegraded":"2024-04-22T07:04:04.859611+0000","last_fullsized":"2024-04-22T07:04:04.859611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:16.859332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859989+0000","last_change":"2024-04-22T07:04:02.790600+0000","last_active":"2024-04-22T07:04:04.859989+0000","last_peered":"2024-04-22T07:04:04.859989+0000","last_clean":"2024-04-22T07:04:04.859989+0000","last_became_active":"2024-04-22T07:04:02.790319+0000","last_became_peered":"2024-04-22T07:04:02.790319+0000","last_unstale":"2024-04-22T07:04:04.859989+0000","last_undegraded":"2024-04-22T07:04:04.859989+0000","last_fullsized":"2024-04-22T07:04:04.859989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787840+0000","last_change":"2024-04-22T07:04:03.788231+0000","last_active":"2024-04-22T07:04:04.787840+0000","last_peered":"2024-04-22T07:04:04.787840+0000","last_clean":"2024-04-22T07:04:04.787840+0000","last_became_active":"2024-04-22T07:04:03.787918+0000","last_became_peered":"2024-04-22T07:04:03.787918+0000","last_unstale":"2024-04-22T07:04:04.787840+0000","last_undegraded":"2024-04-22T07:04:04.787840+0000","last_fullsized":"2024-04-22T07:04:04.787840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861257+0000","last_change":"2024-04-22T07:04:02.796245+0000","last_active":"2024-04-22T07:04:04.861257+0000","last_peered":"2024-04-22T07:04:04.861257+0000","last_clean":"2024-04-22T07:04:04.861257+0000","last_became_active":"2024-04-22T07:04:02.795947+0000","last_became_peered":"2024-04-22T07:04:02.795947+0000","last_unstale":"2024-04-22T07:04:04.861257+0000","last_undegraded":"2024-04-22T07:04:04.861257+0000","last_fullsized":"2024-04-22T07:04:04.861257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790274+0000","last_change":"2024-04-22T07:04:03.793330+0000","last_active":"2024-04-22T07:04:04.790274+0000","last_peered":"2024-04-22T07:04:04.790274+0000","last_clean":"2024-04-22T07:04:04.790274+0000","last_became_active":"2024-04-22T07:04:03.793037+0000","last_became_peered":"2024-04-22T07:04:03.793037+0000","last_unstale":"2024-04-22T07:04:04.790274+0000","last_undegraded":"2024-04-22T07:04:04.790274+0000","last_fullsized":"2024-04-22T07:04:04.790274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:13.368853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787950+0000","last_change":"2024-04-22T07:04:02.793498+0000","last_active":"2024-04-22T07:04:04.787950+0000","last_peered":"2024-04-22T07:04:04.787950+0000","last_clean":"2024-04-22T07:04:04.787950+0000","last_became_active":"2024-04-22T07:04:02.793215+0000","last_became_peered":"2024-04-22T07:04:02.793215+0000","last_unstale":"2024-04-22T07:04:04.787950+0000","last_undegraded":"2024-04-22T07:04:04.787950+0000","last_fullsized":"2024-04-22T07:04:04.787950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860866+0000","last_change":"2024-04-22T07:04:03.804152+0000","last_active":"2024-04-22T07:04:04.860866+0000","last_peered":"2024-04-22T07:04:04.860866+0000","last_clean":"2024-04-22T07:04:04.860866+0000","last_became_active":"2024-04-22T07:04:03.801961+0000","last_became_peered":"2024-04-22T07:04:03.801961+0000","last_unstale":"2024-04-22T07:04:04.860866+0000","last_undegraded":"2024-04-22T07:04:04.860866+0000","last_fullsized":"2024-04-22T07:04:04.860866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862105+0000","last_change":"2024-04-22T07:04:02.793445+0000","last_active":"2024-04-22T07:04:04.862105+0000","last_peered":"2024-04-22T07:04:04.862105+0000","last_clean":"2024-04-22T07:04:04.862105+0000","last_became_active":"2024-04-22T07:04:02.793164+0000","last_became_peered":"2024-04-22T07:04:02.793164+0000","last_unstale":"2024-04-22T07:04:04.862105+0000","last_undegraded":"2024-04-22T07:04:04.862105+0000","last_fullsized":"2024-04-22T07:04:04.862105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788013+0000","last_change":"2024-04-22T07:04:03.789589+0000","last_active":"2024-04-22T07:04:04.788013+0000","last_peered":"2024-04-22T07:04:04.788013+0000","last_clean":"2024-04-22T07:04:04.788013+0000","last_became_active":"2024-04-22T07:04:03.789310+0000","last_became_peered":"2024-04-22T07:04:03.789310+0000","last_unstale":"2024-04-22T07:04:04.788013+0000","last_undegraded":"2024-04-22T07:04:04.788013+0000","last_fullsized":"2024-04-22T07:04:04.788013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788077+0000","last_change":"2024-04-22T07:04:02.789243+0000","last_active":"2024-04-22T07:04:04.788077+0000","last_peered":"2024-04-22T07:04:04.788077+0000","last_clean":"2024-04-22T07:04:04.788077+0000","last_became_active":"2024-04-22T07:04:02.788897+0000","last_became_peered":"2024-04-22T07:04:02.788897+0000","last_unstale":"2024-04-22T07:04:04.788077+0000","last_undegraded":"2024-04-22T07:04:04.788077+0000","last_fullsized":"2024-04-22T07:04:04.788077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860272+0000","last_change":"2024-04-22T07:04:03.793656+0000","last_active":"2024-04-22T07:04:04.860272+0000","last_peered":"2024-04-22T07:04:04.860272+0000","last_clean":"2024-04-22T07:04:04.860272+0000","last_became_active":"2024-04-22T07:04:03.793311+0000","last_became_peered":"2024-04-22T07:04:03.793311+0000","last_unstale":"2024-04-22T07:04:04.860272+0000","last_undegraded":"2024-04-22T07:04:04.860272+0000","last_fullsized":"2024-04-22T07:04:04.860272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860713+0000","last_change":"2024-04-22T07:04:02.786878+0000","last_active":"2024-04-22T07:04:04.860713+0000","last_peered":"2024-04-22T07:04:04.860713+0000","last_clean":"2024-04-22T07:04:04.860713+0000","last_became_active":"2024-04-22T07:04:02.786526+0000","last_became_peered":"2024-04-22T07:04:02.786526+0000","last_unstale":"2024-04-22T07:04:04.860713+0000","last_undegraded":"2024-04-22T07:04:04.860713+0000","last_fullsized":"2024-04-22T07:04:04.860713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860520+0000","last_change":"2024-04-22T07:04:03.791544+0000","last_active":"2024-04-22T07:04:04.860520+0000","last_peered":"2024-04-22T07:04:04.860520+0000","last_clean":"2024-04-22T07:04:04.860520+0000","last_became_active":"2024-04-22T07:04:03.791289+0000","last_became_peered":"2024-04-22T07:04:03.791289+0000","last_unstale":"2024-04-22T07:04:04.860520+0000","last_undegraded":"2024-04-22T07:04:04.860520+0000","last_fullsized":"2024-04-22T07:04:04.860520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859963+0000","last_change":"2024-04-22T07:04:02.795178+0000","last_active":"2024-04-22T07:04:04.859963+0000","last_peered":"2024-04-22T07:04:04.859963+0000","last_clean":"2024-04-22T07:04:04.859963+0000","last_became_active":"2024-04-22T07:04:02.794795+0000","last_became_peered":"2024-04-22T07:04:02.794795+0000","last_unstale":"2024-04-22T07:04:04.859963+0000","last_undegraded":"2024-04-22T07:04:04.859963+0000","last_fullsized":"2024-04-22T07:04:04.859963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790163+0000","last_change":"2024-04-22T07:04:03.788560+0000","last_active":"2024-04-22T07:04:04.790163+0000","last_peered":"2024-04-22T07:04:04.790163+0000","last_clean":"2024-04-22T07:04:04.790163+0000","last_became_active":"2024-04-22T07:04:03.788278+0000","last_became_peered":"2024-04-22T07:04:03.788278+0000","last_unstale":"2024-04-22T07:04:04.790163+0000","last_undegraded":"2024-04-22T07:04:04.790163+0000","last_fullsized":"2024-04-22T07:04:04.790163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:37.492038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787876+0000","last_change":"2024-04-22T07:04:02.788532+0000","last_active":"2024-04-22T07:04:04.787876+0000","last_peered":"2024-04-22T07:04:04.787876+0000","last_clean":"2024-04-22T07:04:04.787876+0000","last_became_active":"2024-04-22T07:04:02.788254+0000","last_became_peered":"2024-04-22T07:04:02.788254+0000","last_unstale":"2024-04-22T07:04:04.787876+0000","last_undegraded":"2024-04-22T07:04:04.787876+0000","last_fullsized":"2024-04-22T07:04:04.787876+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859748+0000","last_change":"2024-04-22T07:04:03.790052+0000","last_active":"2024-04-22T07:04:04.859748+0000","last_peered":"2024-04-22T07:04:04.859748+0000","last_clean":"2024-04-22T07:04:04.859748+0000","last_became_active":"2024-04-22T07:04:03.789351+0000","last_became_peered":"2024-04-22T07:04:03.789351+0000","last_unstale":"2024-04-22T07:04:04.859748+0000","last_undegraded":"2024-04-22T07:04:04.859748+0000","last_fullsized":"2024-04-22T07:04:04.859748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787903+0000","last_change":"2024-04-22T07:04:02.792774+0000","last_active":"2024-04-22T07:04:04.787903+0000","last_peered":"2024-04-22T07:04:04.787903+0000","last_clean":"2024-04-22T07:04:04.787903+0000","last_became_active":"2024-04-22T07:04:02.792282+0000","last_became_peered":"2024-04-22T07:04:02.792282+0000","last_unstale":"2024-04-22T07:04:04.787903+0000","last_undegraded":"2024-04-22T07:04:04.787903+0000","last_fullsized":"2024-04-22T07:04:04.787903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860691+0000","last_change":"2024-04-22T07:04:03.796133+0000","last_active":"2024-04-22T07:04:04.860691+0000","last_peered":"2024-04-22T07:04:04.860691+0000","last_clean":"2024-04-22T07:04:04.860691+0000","last_became_active":"2024-04-22T07:04:03.795854+0000","last_became_peered":"2024-04-22T07:04:03.795854+0000","last_unstale":"2024-04-22T07:04:04.860691+0000","last_undegraded":"2024-04-22T07:04:04.860691+0000","last_fullsized":"2024-04-22T07:04:04.860691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861776+0000","last_change":"2024-04-22T07:04:02.789099+0000","last_active":"2024-04-22T07:04:04.861776+0000","last_peered":"2024-04-22T07:04:04.861776+0000","last_clean":"2024-04-22T07:04:04.861776+0000","last_became_active":"2024-04-22T07:04:02.788782+0000","last_became_peered":"2024-04-22T07:04:02.788782+0000","last_unstale":"2024-04-22T07:04:04.861776+0000","last_undegraded":"2024-04-22T07:04:04.861776+0000","last_fullsized":"2024-04-22T07:04:04.861776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860112+0000","last_change":"2024-04-22T07:04:03.799845+0000","last_active":"2024-04-22T07:04:04.860112+0000","last_peered":"2024-04-22T07:04:04.860112+0000","last_clean":"2024-04-22T07:04:04.860112+0000","last_became_active":"2024-04-22T07:04:03.799553+0000","last_became_peered":"2024-04-22T07:04:03.799553+0000","last_unstale":"2024-04-22T07:04:04.860112+0000","last_undegraded":"2024-04-22T07:04:04.860112+0000","last_fullsized":"2024-04-22T07:04:04.860112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860699+0000","last_change":"2024-04-22T07:04:03.799918+0000","last_active":"2024-04-22T07:04:04.860699+0000","last_peered":"2024-04-22T07:04:04.860699+0000","last_clean":"2024-04-22T07:04:04.860699+0000","last_became_active":"2024-04-22T07:04:03.797851+0000","last_became_peered":"2024-04-22T07:04:03.797851+0000","last_unstale":"2024-04-22T07:04:04.860699+0000","last_undegraded":"2024-04-22T07:04:04.860699+0000","last_fullsized":"2024-04-22T07:04:04.860699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861629+0000","last_change":"2024-04-22T07:04:02.793627+0000","last_active":"2024-04-22T07:04:04.861629+0000","last_peered":"2024-04-22T07:04:04.861629+0000","last_clean":"2024-04-22T07:04:04.861629+0000","last_became_active":"2024-04-22T07:04:02.793341+0000","last_became_peered":"2024-04-22T07:04:02.793341+0000","last_unstale":"2024-04-22T07:04:04.861629+0000","last_undegraded":"2024-04-22T07:04:04.861629+0000","last_fullsized":"2024-04-22T07:04:04.861629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:21:07.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861294+0000","last_change":"2024-04-22T07:04:03.797768+0000","last_active":"2024-04-22T07:04:04.861294+0000","last_peered":"2024-04-22T07:04:04.861294+0000","last_clean":"2024-04-22T07:04:04.861294+0000","last_became_active":"2024-04-22T07:04:03.797437+0000","last_became_peered":"2024-04-22T07:04:03.797437+0000","last_unstale":"2024-04-22T07:04:04.861294+0000","last_undegraded":"2024-04-22T07:04:04.861294+0000","last_fullsized":"2024-04-22T07:04:04.861294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861873+0000","last_change":"2024-04-22T07:04:02.789875+0000","last_active":"2024-04-22T07:04:04.861873+0000","last_peered":"2024-04-22T07:04:04.861873+0000","last_clean":"2024-04-22T07:04:04.861873+0000","last_became_active":"2024-04-22T07:04:02.788989+0000","last_became_peered":"2024-04-22T07:04:02.788989+0000","last_unstale":"2024-04-22T07:04:04.861873+0000","last_undegraded":"2024-04-22T07:04:04.861873+0000","last_fullsized":"2024-04-22T07:04:04.861873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860096+0000","last_change":"2024-04-22T07:04:03.797671+0000","last_active":"2024-04-22T07:04:04.860096+0000","last_peered":"2024-04-22T07:04:04.860096+0000","last_clean":"2024-04-22T07:04:04.860096+0000","last_became_active":"2024-04-22T07:04:03.797399+0000","last_became_peered":"2024-04-22T07:04:03.797399+0000","last_unstale":"2024-04-22T07:04:04.860096+0000","last_undegraded":"2024-04-22T07:04:04.860096+0000","last_fullsized":"2024-04-22T07:04:04.860096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860425+0000","last_change":"2024-04-22T07:04:02.796141+0000","last_active":"2024-04-22T07:04:04.860425+0000","last_peered":"2024-04-22T07:04:04.860425+0000","last_clean":"2024-04-22T07:04:04.860425+0000","last_became_active":"2024-04-22T07:04:02.795821+0000","last_became_peered":"2024-04-22T07:04:02.795821+0000","last_unstale":"2024-04-22T07:04:04.860425+0000","last_undegraded":"2024-04-22T07:04:04.860425+0000","last_fullsized":"2024-04-22T07:04:04.860425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860371+0000","last_change":"2024-04-22T07:04:03.796729+0000","last_active":"2024-04-22T07:04:04.860371+0000","last_peered":"2024-04-22T07:04:04.860371+0000","last_clean":"2024-04-22T07:04:04.860371+0000","last_became_active":"2024-04-22T07:04:03.796486+0000","last_became_peered":"2024-04-22T07:04:03.796486+0000","last_unstale":"2024-04-22T07:04:04.860371+0000","last_undegraded":"2024-04-22T07:04:04.860371+0000","last_fullsized":"2024-04-22T07:04:04.860371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860322+0000","last_change":"2024-04-22T07:04:02.793214+0000","last_active":"2024-04-22T07:04:04.860322+0000","last_peered":"2024-04-22T07:04:04.860322+0000","last_clean":"2024-04-22T07:04:04.860322+0000","last_became_active":"2024-04-22T07:04:02.792922+0000","last_became_peered":"2024-04-22T07:04:02.792922+0000","last_unstale":"2024-04-22T07:04:04.860322+0000","last_undegraded":"2024-04-22T07:04:04.860322+0000","last_fullsized":"2024-04-22T07:04:04.860322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":661302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":661302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:08.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:08.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.885+0000 7f9d48e48640 1 Processor -- start 2024-04-22T07:04:08.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.885+0000 7f9d48e48640 1 -- start start 2024-04-22T07:04:08.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.886+0000 7f9d48e48640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d44152d80 0x7f9d440c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.886+0000 7f9d48e48640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d440c72b0 con 0x7f9d4406fec0 2024-04-22T07:04:08.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.886+0000 7f9d48e48640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d44153160 con 0x7f9d44158de0 2024-04-22T07:04:08.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.886+0000 7f9d48e48640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9d44070300 con 0x7f9d44152d80 2024-04-22T07:04:08.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.886+0000 7f9d41d9b640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9d4406fec0 0x7f9d440ca020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46574/0 (socket says 172.21.15.92:46574) 2024-04-22T07:04:08.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.886+0000 7f9d41d9b640 1 -- 172.21.15.92:0/2433726284 learned_addr learned my addr 172.21.15.92:0/2433726284 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:08.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.886+0000 7f9d4259c640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d44152d80 0x7f9d440c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.886+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 432726412 0 0) 0x7f9d440c72b0 con 0x7f9d4406fec0 2024-04-22T07:04:08.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.886+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d2c003520 con 0x7f9d4406fec0 2024-04-22T07:04:08.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.886+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2253911769 0 0) 0x7f9d2c003520 con 0x7f9d4406fec0 2024-04-22T07:04:08.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d44152d80 msgr2=0x7f9d440c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:08.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4159a640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d44152d80 0x7f9d440c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 >> v1:172.21.15.145:6789/0 conn(0x7f9d44158de0 legacy=0x7f9d44159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d440c6370 con 0x7f9d4406fec0 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f9d34001670 con 0x7f9d4406fec0 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 >> v1:172.21.15.92:6789/0 conn(0x7f9d4406fec0 legacy=0x7f9d440ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4159a640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d44152d80 0x7f9d2c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4159a640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003f20 0x7f9d2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4159a640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d2c004d40 0x7f9d2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d44070300 con 0x7f9d2c003f20 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c00ba20 con 0x7f9d2c004d40 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c00bc20 con 0x7f9d44152d80 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d41d9b640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003f20 0x7f9d2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d4259c640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d44152d80 0x7f9d2c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d42d9d640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d2c004d40 0x7f9d2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d41d9b640 1 -- 172.21.15.92:0/2433726284 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d44152d80 msgr2=0x7f9d2c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:08.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d41d9b640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d44152d80 0x7f9d2c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d41d9b640 1 -- 172.21.15.92:0/2433726284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d2c004d40 msgr2=0x7f9d2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d41d9b640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d2c004d40 0x7f9d2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d41d9b640 1 -- 172.21.15.92:0/2433726284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c003520 con 0x7f9d2c003f20 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.887+0000 7f9d42d9d640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d2c004d40 0x7f9d2c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.888+0000 7f9d41d9b640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003f20 0x7f9d2c004800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400c210 tx=0x7f9d2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433fac5105968dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.888+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d34007ce0 con 0x7f9d2c003f20 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.888+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d34030070 con 0x7f9d2c003f20 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.888+0000 7f9d4159a640 1 -- 172.21.15.92:0/2433726284 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d34005d70 con 0x7f9d2c003f20 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.888+0000 7f9d48e48640 1 -- 172.21.15.92:0/2433726284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003f20 msgr2=0x7f9d2c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.888+0000 7f9d48e48640 1 --2- 172.21.15.92:0/2433726284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003f20 0x7f9d2c004800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400c210 tx=0x7f9d2c005830 comp rx=0 tx=0).stop 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.888+0000 7f9d42d9d640 1 -- 172.21.15.92:0/2433726284 reap_dead start 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.888+0000 7f9d48e48640 1 -- 172.21.15.92:0/2433726284 shutdown_connections 2024-04-22T07:04:08.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.888+0000 7f9d48e48640 1 -- 172.21.15.92:0/2433726284 >> 172.21.15.92:0/2433726284 conn(0x7f9d440cc220 msgr2=0x7f9d440cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:08.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.888+0000 7f9d48e48640 1 -- 172.21.15.92:0/2433726284 shutdown_connections 2024-04-22T07:04:08.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.889+0000 7f9d48e48640 1 -- 172.21.15.92:0/2433726284 wait complete. 2024-04-22T07:04:08.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.890+0000 7f9d48e48640 1 Processor -- start 2024-04-22T07:04:08.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.890+0000 7f9d48e48640 1 -- start start 2024-04-22T07:04:08.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.890+0000 7f9d48e48640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003f20 0x7f9d44141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.890+0000 7f9d48e48640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d441f1280 0x7f9d441f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.890+0000 7f9d48e48640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d441f16c0 0x7f9d441f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.890+0000 7f9d48e48640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d440c5ad0 con 0x7f9d2c003f20 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.890+0000 7f9d48e48640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4406ce60 con 0x7f9d441f1280 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.890+0000 7f9d48e48640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9d440c5db0 con 0x7f9d441f16c0 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d4259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003f20 0x7f9d44141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d4259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003f20 0x7f9d44141080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35970/0 (socket says 172.21.15.92:35970) 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d4259c640 1 -- 172.21.15.92:0/4193081685 learned_addr learned my addr 172.21.15.92:0/4193081685 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d4259c640 1 -- 172.21.15.92:0/4193081685 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d441f16c0 msgr2=0x7f9d441f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d4259c640 1 --2- 172.21.15.92:0/4193081685 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d441f16c0 0x7f9d441f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d4259c640 1 -- 172.21.15.92:0/4193081685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d441f1280 msgr2=0x7f9d441f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d4259c640 1 --2- 172.21.15.92:0/4193081685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d441f1280 0x7f9d441f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d4259c640 1 -- 172.21.15.92:0/4193081685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d44141930 con 0x7f9d2c003f20 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d4259c640 1 --2- 172.21.15.92:0/4193081685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003f20 0x7f9d44141080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9d38011a00 tx=0x7f9d38011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:08.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d32ffd640 1 -- 172.21.15.92:0/4193081685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800fb50 con 0x7f9d2c003f20 2024-04-22T07:04:08.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d32ffd640 1 -- 172.21.15.92:0/4193081685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d3800fcf0 con 0x7f9d2c003f20 2024-04-22T07:04:08.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d48e48640 1 -- 172.21.15.92:0/4193081685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d441eff00 con 0x7f9d2c003f20 2024-04-22T07:04:08.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d32ffd640 1 -- 172.21.15.92:0/4193081685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800fea0 con 0x7f9d2c003f20 2024-04-22T07:04:08.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.891+0000 7f9d48e48640 1 -- 172.21.15.92:0/4193081685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d441f04c0 con 0x7f9d2c003f20 2024-04-22T07:04:08.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.892+0000 7f9d48e48640 1 -- 172.21.15.92:0/4193081685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d44068d30 con 0x7f9d2c003f20 2024-04-22T07:04:08.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.893+0000 7f9d32ffd640 1 -- 172.21.15.92:0/4193081685 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d38012040 con 0x7f9d2c003f20 2024-04-22T07:04:08.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.893+0000 7f9d32ffd640 1 --2- 172.21.15.92:0/4193081685 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9d2406c8f0 0x7f9d2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:08.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.893+0000 7f9d41d9b640 1 --2- 172.21.15.92:0/4193081685 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9d2406c8f0 0x7f9d2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:08.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.893+0000 7f9d32ffd640 1 -- 172.21.15.92:0/4193081685 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9d3808c2f0 con 0x7f9d2c003f20 2024-04-22T07:04:08.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.893+0000 7f9d41d9b640 1 --2- 172.21.15.92:0/4193081685 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9d2406c8f0 0x7f9d2406edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d34034000 tx=0x7f9d34039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:08.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:08.897+0000 7f9d32ffd640 1 -- 172.21.15.92:0/4193081685 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d44068d30 con 0x7f9d2c003f20 2024-04-22T07:04:09.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.071+0000 7f9d48e48640 1 -- 172.21.15.92:0/4193081685 --> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d44069930 con 0x7f9d2406c8f0 2024-04-22T07:04:09.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.077+0000 7f9d32ffd640 1 -- 172.21.15.92:0/4193081685 <== mgr.4107 v2:172.21.15.92:6832/479317737 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251851 (secure 0 0 0) 0x7f9d44069930 con 0x7f9d2406c8f0 2024-04-22T07:04:09.077 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:09.078 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-22T07:04:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.080+0000 7f9d48e48640 1 -- 172.21.15.92:0/4193081685 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9d2406c8f0 msgr2=0x7f9d2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.080+0000 7f9d48e48640 1 --2- 172.21.15.92:0/4193081685 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9d2406c8f0 0x7f9d2406edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d34034000 tx=0x7f9d34039040 comp rx=0 tx=0).stop 2024-04-22T07:04:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.080+0000 7f9d48e48640 1 -- 172.21.15.92:0/4193081685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003f20 msgr2=0x7f9d44141080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.080+0000 7f9d48e48640 1 --2- 172.21.15.92:0/4193081685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003f20 0x7f9d44141080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9d38011a00 tx=0x7f9d38011ec0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.080+0000 7f9d48e48640 1 -- 172.21.15.92:0/4193081685 shutdown_connections 2024-04-22T07:04:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.080+0000 7f9d48e48640 1 --2- 172.21.15.92:0/4193081685 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9d2406c8f0 0x7f9d2406edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.080+0000 7f9d48e48640 1 --2- 172.21.15.92:0/4193081685 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d441f16c0 0x7f9d441f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.080+0000 7f9d48e48640 1 --2- 172.21.15.92:0/4193081685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d441f1280 0x7f9d441f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.080+0000 7f9d48e48640 1 --2- 172.21.15.92:0/4193081685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003f20 0x7f9d44141080 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.080+0000 7f9d48e48640 1 -- 172.21.15.92:0/4193081685 >> 172.21.15.92:0/4193081685 conn(0x7f9d440cc220 msgr2=0x7f9d44151ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.080+0000 7f9d48e48640 1 -- 172.21.15.92:0/4193081685 shutdown_connections 2024-04-22T07:04:09.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.080+0000 7f9d48e48640 1 -- 172.21.15.92:0/4193081685 wait complete. 2024-04-22T07:04:09.088 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-22T07:04:07.933627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218928,"kb_used_data":2760,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755792,"statfs":{"total":773094113280,"available":772869931008,"internally_reserved":0,"allocated":2826240,"data_stored":1748208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.157756"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862171+0000","last_change":"2024-04-22T07:04:03.796924+0000","last_active":"2024-04-22T07:04:04.862171+0000","last_peered":"2024-04-22T07:04:04.862171+0000","last_clean":"2024-04-22T07:04:04.862171+0000","last_became_active":"2024-04-22T07:04:03.796652+0000","last_became_peered":"2024-04-22T07:04:03.796652+0000","last_unstale":"2024-04-22T07:04:04.862171+0000","last_undegraded":"2024-04-22T07:04:04.862171+0000","last_fullsized":"2024-04-22T07:04:04.862171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:01.867906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788117+0000","last_change":"2024-04-22T07:04:03.793652+0000","last_active":"2024-04-22T07:04:04.788117+0000","last_peered":"2024-04-22T07:04:04.788117+0000","last_clean":"2024-04-22T07:04:04.788117+0000","last_became_active":"2024-04-22T07:04:03.793397+0000","last_became_peered":"2024-04-22T07:04:03.793397+0000","last_unstale":"2024-04-22T07:04:04.788117+0000","last_undegraded":"2024-04-22T07:04:04.788117+0000","last_fullsized":"2024-04-22T07:04:04.788117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859908+0000","last_change":"2024-04-22T07:04:03.788534+0000","last_active":"2024-04-22T07:04:04.859908+0000","last_peered":"2024-04-22T07:04:04.859908+0000","last_clean":"2024-04-22T07:04:04.859908+0000","last_became_active":"2024-04-22T07:04:03.787747+0000","last_became_peered":"2024-04-22T07:04:03.787747+0000","last_unstale":"2024-04-22T07:04:04.859908+0000","last_undegraded":"2024-04-22T07:04:04.859908+0000","last_fullsized":"2024-04-22T07:04:04.859908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861337+0000","last_change":"2024-04-22T07:04:03.800150+0000","last_active":"2024-04-22T07:04:04.861337+0000","last_peered":"2024-04-22T07:04:04.861337+0000","last_clean":"2024-04-22T07:04:04.861337+0000","last_became_active":"2024-04-22T07:04:03.799887+0000","last_became_peered":"2024-04-22T07:04:03.799887+0000","last_unstale":"2024-04-22T07:04:04.861337+0000","last_undegraded":"2024-04-22T07:04:04.861337+0000","last_fullsized":"2024-04-22T07:04:04.861337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861727+0000","last_change":"2024-04-22T07:04:03.798482+0000","last_active":"2024-04-22T07:04:04.861727+0000","last_peered":"2024-04-22T07:04:04.861727+0000","last_clean":"2024-04-22T07:04:04.861727+0000","last_became_active":"2024-04-22T07:04:03.798205+0000","last_became_peered":"2024-04-22T07:04:03.798205+0000","last_unstale":"2024-04-22T07:04:04.861727+0000","last_undegraded":"2024-04-22T07:04:04.861727+0000","last_fullsized":"2024-04-22T07:04:04.861727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861044+0000","last_change":"2024-04-22T07:04:03.797373+0000","last_active":"2024-04-22T07:04:04.861044+0000","last_peered":"2024-04-22T07:04:04.861044+0000","last_clean":"2024-04-22T07:04:04.861044+0000","last_became_active":"2024-04-22T07:04:03.797090+0000","last_became_peered":"2024-04-22T07:04:03.797090+0000","last_unstale":"2024-04-22T07:04:04.861044+0000","last_undegraded":"2024-04-22T07:04:04.861044+0000","last_fullsized":"2024-04-22T07:04:04.861044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861437+0000","last_change":"2024-04-22T07:04:03.793725+0000","last_active":"2024-04-22T07:04:04.861437+0000","last_peered":"2024-04-22T07:04:04.861437+0000","last_clean":"2024-04-22T07:04:04.861437+0000","last_became_active":"2024-04-22T07:04:03.793418+0000","last_became_peered":"2024-04-22T07:04:03.793418+0000","last_unstale":"2024-04-22T07:04:04.861437+0000","last_undegraded":"2024-04-22T07:04:04.861437+0000","last_fullsized":"2024-04-22T07:04:04.861437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859500+0000","last_change":"2024-04-22T07:04:03.795377+0000","last_active":"2024-04-22T07:04:04.859500+0000","last_peered":"2024-04-22T07:04:04.859500+0000","last_clean":"2024-04-22T07:04:04.859500+0000","last_became_active":"2024-04-22T07:04:03.795106+0000","last_became_peered":"2024-04-22T07:04:03.795106+0000","last_unstale":"2024-04-22T07:04:04.859500+0000","last_undegraded":"2024-04-22T07:04:04.859500+0000","last_fullsized":"2024-04-22T07:04:04.859500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862209+0000","last_change":"2024-04-22T07:04:03.797692+0000","last_active":"2024-04-22T07:04:04.862209+0000","last_peered":"2024-04-22T07:04:04.862209+0000","last_clean":"2024-04-22T07:04:04.862209+0000","last_became_active":"2024-04-22T07:04:03.797396+0000","last_became_peered":"2024-04-22T07:04:03.797396+0000","last_unstale":"2024-04-22T07:04:04.862209+0000","last_undegraded":"2024-04-22T07:04:04.862209+0000","last_fullsized":"2024-04-22T07:04:04.862209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:39.068857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860742+0000","last_change":"2024-04-22T07:04:03.797228+0000","last_active":"2024-04-22T07:04:04.860742+0000","last_peered":"2024-04-22T07:04:04.860742+0000","last_clean":"2024-04-22T07:04:04.860742+0000","last_became_active":"2024-04-22T07:04:03.796614+0000","last_became_peered":"2024-04-22T07:04:03.796614+0000","last_unstale":"2024-04-22T07:04:04.860742+0000","last_undegraded":"2024-04-22T07:04:04.860742+0000","last_fullsized":"2024-04-22T07:04:04.860742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860497+0000","last_change":"2024-04-22T07:04:03.799434+0000","last_active":"2024-04-22T07:04:04.860497+0000","last_peered":"2024-04-22T07:04:04.860497+0000","last_clean":"2024-04-22T07:04:04.860497+0000","last_became_active":"2024-04-22T07:04:03.799173+0000","last_became_peered":"2024-04-22T07:04:03.799173+0000","last_unstale":"2024-04-22T07:04:04.860497+0000","last_undegraded":"2024-04-22T07:04:04.860497+0000","last_fullsized":"2024-04-22T07:04:04.860497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790488+0000","last_change":"2024-04-22T07:04:03.800035+0000","last_active":"2024-04-22T07:04:04.790488+0000","last_peered":"2024-04-22T07:04:04.790488+0000","last_clean":"2024-04-22T07:04:04.790488+0000","last_became_active":"2024-04-22T07:04:03.799646+0000","last_became_peered":"2024-04-22T07:04:03.799646+0000","last_unstale":"2024-04-22T07:04:04.790488+0000","last_undegraded":"2024-04-22T07:04:04.790488+0000","last_fullsized":"2024-04-22T07:04:04.790488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:17.287068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860651+0000","last_change":"2024-04-22T07:04:03.797324+0000","last_active":"2024-04-22T07:04:04.860651+0000","last_peered":"2024-04-22T07:04:04.860651+0000","last_clean":"2024-04-22T07:04:04.860651+0000","last_became_active":"2024-04-22T07:04:03.797039+0000","last_became_peered":"2024-04-22T07:04:03.797039+0000","last_unstale":"2024-04-22T07:04:04.860651+0000","last_undegraded":"2024-04-22T07:04:04.860651+0000","last_fullsized":"2024-04-22T07:04:04.860651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859720+0000","last_change":"2024-04-22T07:04:03.790732+0000","last_active":"2024-04-22T07:04:04.859720+0000","last_peered":"2024-04-22T07:04:04.859720+0000","last_clean":"2024-04-22T07:04:04.859720+0000","last_became_active":"2024-04-22T07:04:03.788340+0000","last_became_peered":"2024-04-22T07:04:03.788340+0000","last_unstale":"2024-04-22T07:04:04.859720+0000","last_undegraded":"2024-04-22T07:04:04.859720+0000","last_fullsized":"2024-04-22T07:04:04.859720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787860+0000","last_change":"2024-04-22T07:04:03.799368+0000","last_active":"2024-04-22T07:04:04.787860+0000","last_peered":"2024-04-22T07:04:04.787860+0000","last_clean":"2024-04-22T07:04:04.787860+0000","last_became_active":"2024-04-22T07:04:03.799067+0000","last_became_peered":"2024-04-22T07:04:03.799067+0000","last_unstale":"2024-04-22T07:04:04.787860+0000","last_undegraded":"2024-04-22T07:04:04.787860+0000","last_fullsized":"2024-04-22T07:04:04.787860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860988+0000","last_change":"2024-04-22T07:04:03.802947+0000","last_active":"2024-04-22T07:04:04.860988+0000","last_peered":"2024-04-22T07:04:04.860988+0000","last_clean":"2024-04-22T07:04:04.860988+0000","last_became_active":"2024-04-22T07:04:03.802493+0000","last_became_peered":"2024-04-22T07:04:03.802493+0000","last_unstale":"2024-04-22T07:04:04.860988+0000","last_undegraded":"2024-04-22T07:04:04.860988+0000","last_fullsized":"2024-04-22T07:04:04.860988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860714+0000","last_change":"2024-04-22T07:04:03.795581+0000","last_active":"2024-04-22T07:04:04.860714+0000","last_peered":"2024-04-22T07:04:04.860714+0000","last_clean":"2024-04-22T07:04:04.860714+0000","last_became_active":"2024-04-22T07:04:03.795292+0000","last_became_peered":"2024-04-22T07:04:03.795292+0000","last_unstale":"2024-04-22T07:04:04.860714+0000","last_undegraded":"2024-04-22T07:04:04.860714+0000","last_fullsized":"2024-04-22T07:04:04.860714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859993+0000","last_change":"2024-04-22T07:04:03.796132+0000","last_active":"2024-04-22T07:04:04.859993+0000","last_peered":"2024-04-22T07:04:04.859993+0000","last_clean":"2024-04-22T07:04:04.859993+0000","last_became_active":"2024-04-22T07:04:03.795069+0000","last_became_peered":"2024-04-22T07:04:03.795069+0000","last_unstale":"2024-04-22T07:04:04.859993+0000","last_undegraded":"2024-04-22T07:04:04.859993+0000","last_fullsized":"2024-04-22T07:04:04.859993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:08.934970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862125+0000","last_change":"2024-04-22T07:04:03.794552+0000","last_active":"2024-04-22T07:04:04.862125+0000","last_peered":"2024-04-22T07:04:04.862125+0000","last_clean":"2024-04-22T07:04:04.862125+0000","last_became_active":"2024-04-22T07:04:03.794046+0000","last_became_peered":"2024-04-22T07:04:03.794046+0000","last_unstale":"2024-04-22T07:04:04.862125+0000","last_undegraded":"2024-04-22T07:04:04.862125+0000","last_fullsized":"2024-04-22T07:04:04.862125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860703+0000","last_change":"2024-04-22T07:04:03.798686+0000","last_active":"2024-04-22T07:04:04.860703+0000","last_peered":"2024-04-22T07:04:04.860703+0000","last_clean":"2024-04-22T07:04:04.860703+0000","last_became_active":"2024-04-22T07:04:03.798104+0000","last_became_peered":"2024-04-22T07:04:03.798104+0000","last_unstale":"2024-04-22T07:04:04.860703+0000","last_undegraded":"2024-04-22T07:04:04.860703+0000","last_fullsized":"2024-04-22T07:04:04.860703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860494+0000","last_change":"2024-04-22T07:04:03.794215+0000","last_active":"2024-04-22T07:04:04.860494+0000","last_peered":"2024-04-22T07:04:04.860494+0000","last_clean":"2024-04-22T07:04:04.860494+0000","last_became_active":"2024-04-22T07:04:03.793990+0000","last_became_peered":"2024-04-22T07:04:03.793990+0000","last_unstale":"2024-04-22T07:04:04.860494+0000","last_undegraded":"2024-04-22T07:04:04.860494+0000","last_fullsized":"2024-04-22T07:04:04.860494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861162+0000","last_change":"2024-04-22T07:04:03.798539+0000","last_active":"2024-04-22T07:04:04.861162+0000","last_peered":"2024-04-22T07:04:04.861162+0000","last_clean":"2024-04-22T07:04:04.861162+0000","last_became_active":"2024-04-22T07:04:03.797713+0000","last_became_peered":"2024-04-22T07:04:03.797713+0000","last_unstale":"2024-04-22T07:04:04.861162+0000","last_undegraded":"2024-04-22T07:04:04.861162+0000","last_fullsized":"2024-04-22T07:04:04.861162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859945+0000","last_change":"2024-04-22T07:04:03.798610+0000","last_active":"2024-04-22T07:04:04.859945+0000","last_peered":"2024-04-22T07:04:04.859945+0000","last_clean":"2024-04-22T07:04:04.859945+0000","last_became_active":"2024-04-22T07:04:03.797981+0000","last_became_peered":"2024-04-22T07:04:03.797981+0000","last_unstale":"2024-04-22T07:04:04.859945+0000","last_undegraded":"2024-04-22T07:04:04.859945+0000","last_fullsized":"2024-04-22T07:04:04.859945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861507+0000","last_change":"2024-04-22T07:04:03.802742+0000","last_active":"2024-04-22T07:04:04.861507+0000","last_peered":"2024-04-22T07:04:04.861507+0000","last_clean":"2024-04-22T07:04:04.861507+0000","last_became_active":"2024-04-22T07:04:03.802127+0000","last_became_peered":"2024-04-22T07:04:03.802127+0000","last_unstale":"2024-04-22T07:04:04.861507+0000","last_undegraded":"2024-04-22T07:04:04.861507+0000","last_fullsized":"2024-04-22T07:04:04.861507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860138+0000","last_change":"2024-04-22T07:04:03.798789+0000","last_active":"2024-04-22T07:04:04.860138+0000","last_peered":"2024-04-22T07:04:04.860138+0000","last_clean":"2024-04-22T07:04:04.860138+0000","last_became_active":"2024-04-22T07:04:03.798505+0000","last_became_peered":"2024-04-22T07:04:03.798505+0000","last_unstale":"2024-04-22T07:04:04.860138+0000","last_undegraded":"2024-04-22T07:04:04.860138+0000","last_fullsized":"2024-04-22T07:04:04.860138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860919+0000","last_change":"2024-04-22T07:04:03.794111+0000","last_active":"2024-04-22T07:04:04.860919+0000","last_peered":"2024-04-22T07:04:04.860919+0000","last_clean":"2024-04-22T07:04:04.860919+0000","last_became_active":"2024-04-22T07:04:03.793682+0000","last_became_peered":"2024-04-22T07:04:03.793682+0000","last_unstale":"2024-04-22T07:04:04.860919+0000","last_undegraded":"2024-04-22T07:04:04.860919+0000","last_fullsized":"2024-04-22T07:04:04.860919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788083+0000","last_change":"2024-04-22T07:04:03.790167+0000","last_active":"2024-04-22T07:04:04.788083+0000","last_peered":"2024-04-22T07:04:04.788083+0000","last_clean":"2024-04-22T07:04:04.788083+0000","last_became_active":"2024-04-22T07:04:03.789873+0000","last_became_peered":"2024-04-22T07:04:03.789873+0000","last_unstale":"2024-04-22T07:04:04.788083+0000","last_undegraded":"2024-04-22T07:04:04.788083+0000","last_fullsized":"2024-04-22T07:04:04.788083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859954+0000","last_change":"2024-04-22T07:04:03.798713+0000","last_active":"2024-04-22T07:04:04.859954+0000","last_peered":"2024-04-22T07:04:04.859954+0000","last_clean":"2024-04-22T07:04:04.859954+0000","last_became_active":"2024-04-22T07:04:03.798344+0000","last_became_peered":"2024-04-22T07:04:03.798344+0000","last_unstale":"2024-04-22T07:04:04.859954+0000","last_undegraded":"2024-04-22T07:04:04.859954+0000","last_fullsized":"2024-04-22T07:04:04.859954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860634+0000","last_change":"2024-04-22T07:04:03.795452+0000","last_active":"2024-04-22T07:04:04.860634+0000","last_peered":"2024-04-22T07:04:04.860634+0000","last_clean":"2024-04-22T07:04:04.860634+0000","last_became_active":"2024-04-22T07:04:03.794970+0000","last_became_peered":"2024-04-22T07:04:03.794970+0000","last_unstale":"2024-04-22T07:04:04.860634+0000","last_undegraded":"2024-04-22T07:04:04.860634+0000","last_fullsized":"2024-04-22T07:04:04.860634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861203+0000","last_change":"2024-04-22T07:04:03.799581+0000","last_active":"2024-04-22T07:04:04.861203+0000","last_peered":"2024-04-22T07:04:04.861203+0000","last_clean":"2024-04-22T07:04:04.861203+0000","last_became_active":"2024-04-22T07:04:03.799328+0000","last_became_peered":"2024-04-22T07:04:03.799328+0000","last_unstale":"2024-04-22T07:04:04.861203+0000","last_undegraded":"2024-04-22T07:04:04.861203+0000","last_fullsized":"2024-04-22T07:04:04.861203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859944+0000","last_change":"2024-04-22T07:04:03.796830+0000","last_active":"2024-04-22T07:04:04.859944+0000","last_peered":"2024-04-22T07:04:04.859944+0000","last_clean":"2024-04-22T07:04:04.859944+0000","last_became_active":"2024-04-22T07:04:03.796551+0000","last_became_peered":"2024-04-22T07:04:03.796551+0000","last_unstale":"2024-04-22T07:04:04.859944+0000","last_undegraded":"2024-04-22T07:04:04.859944+0000","last_fullsized":"2024-04-22T07:04:04.859944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860101+0000","last_change":"2024-04-22T07:04:03.795320+0000","last_active":"2024-04-22T07:04:04.860101+0000","last_peered":"2024-04-22T07:04:04.860101+0000","last_clean":"2024-04-22T07:04:04.860101+0000","last_became_active":"2024-04-22T07:04:03.795025+0000","last_became_peered":"2024-04-22T07:04:03.795025+0000","last_unstale":"2024-04-22T07:04:04.860101+0000","last_undegraded":"2024-04-22T07:04:04.860101+0000","last_fullsized":"2024-04-22T07:04:04.860101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790119+0000","last_change":"2024-04-22T07:04:03.788859+0000","last_active":"2024-04-22T07:04:04.790119+0000","last_peered":"2024-04-22T07:04:04.790119+0000","last_clean":"2024-04-22T07:04:04.790119+0000","last_became_active":"2024-04-22T07:04:03.788573+0000","last_became_peered":"2024-04-22T07:04:03.788573+0000","last_unstale":"2024-04-22T07:04:04.790119+0000","last_undegraded":"2024-04-22T07:04:04.790119+0000","last_fullsized":"2024-04-22T07:04:04.790119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:36.716689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860694+0000","last_change":"2024-04-22T07:04:02.795724+0000","last_active":"2024-04-22T07:04:04.860694+0000","last_peered":"2024-04-22T07:04:04.860694+0000","last_clean":"2024-04-22T07:04:04.860694+0000","last_became_active":"2024-04-22T07:04:02.795371+0000","last_became_peered":"2024-04-22T07:04:02.795371+0000","last_unstale":"2024-04-22T07:04:04.860694+0000","last_undegraded":"2024-04-22T07:04:04.860694+0000","last_fullsized":"2024-04-22T07:04:04.860694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860689+0000","last_change":"2024-04-22T07:04:03.787030+0000","last_active":"2024-04-22T07:04:04.860689+0000","last_peered":"2024-04-22T07:04:04.860689+0000","last_clean":"2024-04-22T07:04:04.860689+0000","last_became_active":"2024-04-22T07:04:03.786357+0000","last_became_peered":"2024-04-22T07:04:03.786357+0000","last_unstale":"2024-04-22T07:04:04.860689+0000","last_undegraded":"2024-04-22T07:04:04.860689+0000","last_fullsized":"2024-04-22T07:04:04.860689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860316+0000","last_change":"2024-04-22T07:04:02.791429+0000","last_active":"2024-04-22T07:04:04.860316+0000","last_peered":"2024-04-22T07:04:04.860316+0000","last_clean":"2024-04-22T07:04:04.860316+0000","last_became_active":"2024-04-22T07:04:02.791114+0000","last_became_peered":"2024-04-22T07:04:02.791114+0000","last_unstale":"2024-04-22T07:04:04.860316+0000","last_undegraded":"2024-04-22T07:04:04.860316+0000","last_fullsized":"2024-04-22T07:04:04.860316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860040+0000","last_change":"2024-04-22T07:04:02.789531+0000","last_active":"2024-04-22T07:04:04.860040+0000","last_peered":"2024-04-22T07:04:04.860040+0000","last_clean":"2024-04-22T07:04:04.860040+0000","last_became_active":"2024-04-22T07:04:02.789248+0000","last_became_peered":"2024-04-22T07:04:02.789248+0000","last_unstale":"2024-04-22T07:04:04.860040+0000","last_undegraded":"2024-04-22T07:04:04.860040+0000","last_fullsized":"2024-04-22T07:04:04.860040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:06:43.031782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859624+0000","last_change":"2024-04-22T07:04:03.790142+0000","last_active":"2024-04-22T07:04:04.859624+0000","last_peered":"2024-04-22T07:04:04.859624+0000","last_clean":"2024-04-22T07:04:04.859624+0000","last_became_active":"2024-04-22T07:04:03.789869+0000","last_became_peered":"2024-04-22T07:04:03.789869+0000","last_unstale":"2024-04-22T07:04:04.859624+0000","last_undegraded":"2024-04-22T07:04:04.859624+0000","last_fullsized":"2024-04-22T07:04:04.859624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861050+0000","last_change":"2024-04-22T07:04:02.790537+0000","last_active":"2024-04-22T07:04:04.861050+0000","last_peered":"2024-04-22T07:04:04.861050+0000","last_clean":"2024-04-22T07:04:04.861050+0000","last_became_active":"2024-04-22T07:04:02.789072+0000","last_became_peered":"2024-04-22T07:04:02.789072+0000","last_unstale":"2024-04-22T07:04:04.861050+0000","last_undegraded":"2024-04-22T07:04:04.861050+0000","last_fullsized":"2024-04-22T07:04:04.861050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787835+0000","last_change":"2024-04-22T07:04:03.791239+0000","last_active":"2024-04-22T07:04:04.787835+0000","last_peered":"2024-04-22T07:04:04.787835+0000","last_clean":"2024-04-22T07:04:04.787835+0000","last_became_active":"2024-04-22T07:04:03.790515+0000","last_became_peered":"2024-04-22T07:04:03.790515+0000","last_unstale":"2024-04-22T07:04:04.787835+0000","last_undegraded":"2024-04-22T07:04:04.787835+0000","last_fullsized":"2024-04-22T07:04:04.787835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860666+0000","last_change":"2024-04-22T07:04:02.793582+0000","last_active":"2024-04-22T07:04:04.860666+0000","last_peered":"2024-04-22T07:04:04.860666+0000","last_clean":"2024-04-22T07:04:04.860666+0000","last_became_active":"2024-04-22T07:04:02.793281+0000","last_became_peered":"2024-04-22T07:04:02.793281+0000","last_unstale":"2024-04-22T07:04:04.860666+0000","last_undegraded":"2024-04-22T07:04:04.860666+0000","last_fullsized":"2024-04-22T07:04:04.860666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860510+0000","last_change":"2024-04-22T07:04:03.793994+0000","last_active":"2024-04-22T07:04:04.860510+0000","last_peered":"2024-04-22T07:04:04.860510+0000","last_clean":"2024-04-22T07:04:04.860510+0000","last_became_active":"2024-04-22T07:04:03.793734+0000","last_became_peered":"2024-04-22T07:04:03.793734+0000","last_unstale":"2024-04-22T07:04:04.860510+0000","last_undegraded":"2024-04-22T07:04:04.860510+0000","last_fullsized":"2024-04-22T07:04:04.860510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860572+0000","last_change":"2024-04-22T07:04:02.791945+0000","last_active":"2024-04-22T07:04:04.860572+0000","last_peered":"2024-04-22T07:04:04.860572+0000","last_clean":"2024-04-22T07:04:04.860572+0000","last_became_active":"2024-04-22T07:04:02.791654+0000","last_became_peered":"2024-04-22T07:04:02.791654+0000","last_unstale":"2024-04-22T07:04:04.860572+0000","last_undegraded":"2024-04-22T07:04:04.860572+0000","last_fullsized":"2024-04-22T07:04:04.860572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860230+0000","last_change":"2024-04-22T07:04:03.797391+0000","last_active":"2024-04-22T07:04:04.860230+0000","last_peered":"2024-04-22T07:04:04.860230+0000","last_clean":"2024-04-22T07:04:04.860230+0000","last_became_active":"2024-04-22T07:04:03.797123+0000","last_became_peered":"2024-04-22T07:04:03.797123+0000","last_unstale":"2024-04-22T07:04:04.860230+0000","last_undegraded":"2024-04-22T07:04:04.860230+0000","last_fullsized":"2024-04-22T07:04:04.860230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788045+0000","last_change":"2024-04-22T07:04:02.793753+0000","last_active":"2024-04-22T07:04:04.788045+0000","last_peered":"2024-04-22T07:04:04.788045+0000","last_clean":"2024-04-22T07:04:04.788045+0000","last_became_active":"2024-04-22T07:04:02.793436+0000","last_became_peered":"2024-04-22T07:04:02.793436+0000","last_unstale":"2024-04-22T07:04:04.788045+0000","last_undegraded":"2024-04-22T07:04:04.788045+0000","last_fullsized":"2024-04-22T07:04:04.788045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790527+0000","last_change":"2024-04-22T07:04:03.800665+0000","last_active":"2024-04-22T07:04:04.790527+0000","last_peered":"2024-04-22T07:04:04.790527+0000","last_clean":"2024-04-22T07:04:04.790527+0000","last_became_active":"2024-04-22T07:04:03.800227+0000","last_became_peered":"2024-04-22T07:04:03.800227+0000","last_unstale":"2024-04-22T07:04:04.790527+0000","last_undegraded":"2024-04-22T07:04:04.790527+0000","last_fullsized":"2024-04-22T07:04:04.790527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860760+0000","last_change":"2024-04-22T07:04:02.791309+0000","last_active":"2024-04-22T07:04:04.860760+0000","last_peered":"2024-04-22T07:04:04.860760+0000","last_clean":"2024-04-22T07:04:04.860760+0000","last_became_active":"2024-04-22T07:04:02.791000+0000","last_became_peered":"2024-04-22T07:04:02.791000+0000","last_unstale":"2024-04-22T07:04:04.860760+0000","last_undegraded":"2024-04-22T07:04:04.860760+0000","last_fullsized":"2024-04-22T07:04:04.860760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861543+0000","last_change":"2024-04-22T07:04:03.794316+0000","last_active":"2024-04-22T07:04:04.861543+0000","last_peered":"2024-04-22T07:04:04.861543+0000","last_clean":"2024-04-22T07:04:04.861543+0000","last_became_active":"2024-04-22T07:04:03.793646+0000","last_became_peered":"2024-04-22T07:04:03.793646+0000","last_unstale":"2024-04-22T07:04:04.861543+0000","last_undegraded":"2024-04-22T07:04:04.861543+0000","last_fullsized":"2024-04-22T07:04:04.861543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860673+0000","last_change":"2024-04-22T07:04:02.791024+0000","last_active":"2024-04-22T07:04:04.860673+0000","last_peered":"2024-04-22T07:04:04.860673+0000","last_clean":"2024-04-22T07:04:04.860673+0000","last_became_active":"2024-04-22T07:04:02.789160+0000","last_became_peered":"2024-04-22T07:04:02.789160+0000","last_unstale":"2024-04-22T07:04:04.860673+0000","last_undegraded":"2024-04-22T07:04:04.860673+0000","last_fullsized":"2024-04-22T07:04:04.860673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859792+0000","last_change":"2024-04-22T07:04:03.798436+0000","last_active":"2024-04-22T07:04:04.859792+0000","last_peered":"2024-04-22T07:04:04.859792+0000","last_clean":"2024-04-22T07:04:04.859792+0000","last_became_active":"2024-04-22T07:04:03.797946+0000","last_became_peered":"2024-04-22T07:04:03.797946+0000","last_unstale":"2024-04-22T07:04:04.859792+0000","last_undegraded":"2024-04-22T07:04:04.859792+0000","last_fullsized":"2024-04-22T07:04:04.859792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861234+0000","last_change":"2024-04-22T07:04:02.796551+0000","last_active":"2024-04-22T07:04:04.861234+0000","last_peered":"2024-04-22T07:04:04.861234+0000","last_clean":"2024-04-22T07:04:04.861234+0000","last_became_active":"2024-04-22T07:04:02.796332+0000","last_became_peered":"2024-04-22T07:04:02.796332+0000","last_unstale":"2024-04-22T07:04:04.861234+0000","last_undegraded":"2024-04-22T07:04:04.861234+0000","last_fullsized":"2024-04-22T07:04:04.861234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:15.854295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861758+0000","last_change":"2024-04-22T07:04:03.804622+0000","last_active":"2024-04-22T07:04:04.861758+0000","last_peered":"2024-04-22T07:04:04.861758+0000","last_clean":"2024-04-22T07:04:04.861758+0000","last_became_active":"2024-04-22T07:04:03.804324+0000","last_became_peered":"2024-04-22T07:04:03.804324+0000","last_unstale":"2024-04-22T07:04:04.861758+0000","last_undegraded":"2024-04-22T07:04:04.861758+0000","last_fullsized":"2024-04-22T07:04:04.861758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860167+0000","last_change":"2024-04-22T07:04:02.787085+0000","last_active":"2024-04-22T07:04:04.860167+0000","last_peered":"2024-04-22T07:04:04.860167+0000","last_clean":"2024-04-22T07:04:04.860167+0000","last_became_active":"2024-04-22T07:04:02.786777+0000","last_became_peered":"2024-04-22T07:04:02.786777+0000","last_unstale":"2024-04-22T07:04:04.860167+0000","last_undegraded":"2024-04-22T07:04:04.860167+0000","last_fullsized":"2024-04-22T07:04:04.860167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860535+0000","last_change":"2024-04-22T07:04:03.796356+0000","last_active":"2024-04-22T07:04:04.860535+0000","last_peered":"2024-04-22T07:04:04.860535+0000","last_clean":"2024-04-22T07:04:04.860535+0000","last_became_active":"2024-04-22T07:04:03.796030+0000","last_became_peered":"2024-04-22T07:04:03.796030+0000","last_unstale":"2024-04-22T07:04:04.860535+0000","last_undegraded":"2024-04-22T07:04:04.860535+0000","last_fullsized":"2024-04-22T07:04:04.860535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860233+0000","last_change":"2024-04-22T07:04:02.787425+0000","last_active":"2024-04-22T07:04:04.860233+0000","last_peered":"2024-04-22T07:04:04.860233+0000","last_clean":"2024-04-22T07:04:04.860233+0000","last_became_active":"2024-04-22T07:04:02.787146+0000","last_became_peered":"2024-04-22T07:04:02.787146+0000","last_unstale":"2024-04-22T07:04:04.860233+0000","last_undegraded":"2024-04-22T07:04:04.860233+0000","last_fullsized":"2024-04-22T07:04:04.860233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861631+0000","last_change":"2024-04-22T07:04:03.802924+0000","last_active":"2024-04-22T07:04:04.861631+0000","last_peered":"2024-04-22T07:04:04.861631+0000","last_clean":"2024-04-22T07:04:04.861631+0000","last_became_active":"2024-04-22T07:04:03.802644+0000","last_became_peered":"2024-04-22T07:04:03.802644+0000","last_unstale":"2024-04-22T07:04:04.861631+0000","last_undegraded":"2024-04-22T07:04:04.861631+0000","last_fullsized":"2024-04-22T07:04:04.861631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790380+0000","last_change":"2024-04-22T07:04:02.794481+0000","last_active":"2024-04-22T07:04:04.790380+0000","last_peered":"2024-04-22T07:04:04.790380+0000","last_clean":"2024-04-22T07:04:04.790380+0000","last_became_active":"2024-04-22T07:04:02.793982+0000","last_became_peered":"2024-04-22T07:04:02.793982+0000","last_unstale":"2024-04-22T07:04:04.790380+0000","last_undegraded":"2024-04-22T07:04:04.790380+0000","last_fullsized":"2024-04-22T07:04:04.790380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:58.906416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862139+0000","last_change":"2024-04-22T07:04:03.802724+0000","last_active":"2024-04-22T07:04:04.862139+0000","last_peered":"2024-04-22T07:04:04.862139+0000","last_clean":"2024-04-22T07:04:04.862139+0000","last_became_active":"2024-04-22T07:04:03.802434+0000","last_became_peered":"2024-04-22T07:04:03.802434+0000","last_unstale":"2024-04-22T07:04:04.862139+0000","last_undegraded":"2024-04-22T07:04:04.862139+0000","last_fullsized":"2024-04-22T07:04:04.862139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860035+0000","last_change":"2024-04-22T07:04:02.791375+0000","last_active":"2024-04-22T07:04:04.860035+0000","last_peered":"2024-04-22T07:04:04.860035+0000","last_clean":"2024-04-22T07:04:04.860035+0000","last_became_active":"2024-04-22T07:04:02.791095+0000","last_became_peered":"2024-04-22T07:04:02.791095+0000","last_unstale":"2024-04-22T07:04:04.860035+0000","last_undegraded":"2024-04-22T07:04:04.860035+0000","last_fullsized":"2024-04-22T07:04:04.860035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862265+0000","last_change":"2024-04-22T07:04:03.802028+0000","last_active":"2024-04-22T07:04:04.862265+0000","last_peered":"2024-04-22T07:04:04.862265+0000","last_clean":"2024-04-22T07:04:04.862265+0000","last_became_active":"2024-04-22T07:04:03.801715+0000","last_became_peered":"2024-04-22T07:04:03.801715+0000","last_unstale":"2024-04-22T07:04:04.862265+0000","last_undegraded":"2024-04-22T07:04:04.862265+0000","last_fullsized":"2024-04-22T07:04:04.862265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:29.815068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788119+0000","last_change":"2024-04-22T07:04:02.793270+0000","last_active":"2024-04-22T07:04:04.788119+0000","last_peered":"2024-04-22T07:04:04.788119+0000","last_clean":"2024-04-22T07:04:04.788119+0000","last_became_active":"2024-04-22T07:04:02.793009+0000","last_became_peered":"2024-04-22T07:04:02.793009+0000","last_unstale":"2024-04-22T07:04:04.788119+0000","last_undegraded":"2024-04-22T07:04:04.788119+0000","last_fullsized":"2024-04-22T07:04:04.788119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861459+0000","last_change":"2024-04-22T07:04:03.800175+0000","last_active":"2024-04-22T07:04:04.861459+0000","last_peered":"2024-04-22T07:04:04.861459+0000","last_clean":"2024-04-22T07:04:04.861459+0000","last_became_active":"2024-04-22T07:04:03.799902+0000","last_became_peered":"2024-04-22T07:04:03.799902+0000","last_unstale":"2024-04-22T07:04:04.861459+0000","last_undegraded":"2024-04-22T07:04:04.861459+0000","last_fullsized":"2024-04-22T07:04:04.861459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859806+0000","last_change":"2024-04-22T07:04:02.793643+0000","last_active":"2024-04-22T07:04:04.859806+0000","last_peered":"2024-04-22T07:04:04.859806+0000","last_clean":"2024-04-22T07:04:04.859806+0000","last_became_active":"2024-04-22T07:04:02.793333+0000","last_became_peered":"2024-04-22T07:04:02.793333+0000","last_unstale":"2024-04-22T07:04:04.859806+0000","last_undegraded":"2024-04-22T07:04:04.859806+0000","last_fullsized":"2024-04-22T07:04:04.859806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862019+0000","last_change":"2024-04-22T07:04:03.801821+0000","last_active":"2024-04-22T07:04:04.862019+0000","last_peered":"2024-04-22T07:04:04.862019+0000","last_clean":"2024-04-22T07:04:04.862019+0000","last_became_active":"2024-04-22T07:04:03.801510+0000","last_became_peered":"2024-04-22T07:04:03.801510+0000","last_unstale":"2024-04-22T07:04:04.862019+0000","last_undegraded":"2024-04-22T07:04:04.862019+0000","last_fullsized":"2024-04-22T07:04:04.862019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861241+0000","last_change":"2024-04-22T07:03:59.215285+0000","last_active":"2024-04-22T07:04:04.861241+0000","last_peered":"2024-04-22T07:04:04.861241+0000","last_clean":"2024-04-22T07:04:04.861241+0000","last_became_active":"2024-04-22T07:03:59.214275+0000","last_became_peered":"2024-04-22T07:03:59.214275+0000","last_unstale":"2024-04-22T07:04:04.861241+0000","last_undegraded":"2024-04-22T07:04:04.861241+0000","last_fullsized":"2024-04-22T07:04:04.861241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_clean_scrub_stamp":"2024-04-22T07:03:57.959761+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:55.105073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860226+0000","last_change":"2024-04-22T07:04:02.793384+0000","last_active":"2024-04-22T07:04:04.860226+0000","last_peered":"2024-04-22T07:04:04.860226+0000","last_clean":"2024-04-22T07:04:04.860226+0000","last_became_active":"2024-04-22T07:04:02.793077+0000","last_became_peered":"2024-04-22T07:04:02.793077+0000","last_unstale":"2024-04-22T07:04:04.860226+0000","last_undegraded":"2024-04-22T07:04:04.860226+0000","last_fullsized":"2024-04-22T07:04:04.860226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860779+0000","last_change":"2024-04-22T07:04:03.798134+0000","last_active":"2024-04-22T07:04:04.860779+0000","last_peered":"2024-04-22T07:04:04.860779+0000","last_clean":"2024-04-22T07:04:04.860779+0000","last_became_active":"2024-04-22T07:04:03.797848+0000","last_became_peered":"2024-04-22T07:04:03.797848+0000","last_unstale":"2024-04-22T07:04:04.860779+0000","last_undegraded":"2024-04-22T07:04:04.860779+0000","last_fullsized":"2024-04-22T07:04:04.860779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859966+0000","last_change":"2024-04-22T07:04:02.789591+0000","last_active":"2024-04-22T07:04:04.859966+0000","last_peered":"2024-04-22T07:04:04.859966+0000","last_clean":"2024-04-22T07:04:04.859966+0000","last_became_active":"2024-04-22T07:04:02.788785+0000","last_became_peered":"2024-04-22T07:04:02.788785+0000","last_unstale":"2024-04-22T07:04:04.859966+0000","last_undegraded":"2024-04-22T07:04:04.859966+0000","last_fullsized":"2024-04-22T07:04:04.859966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859611+0000","last_change":"2024-04-22T07:04:03.789757+0000","last_active":"2024-04-22T07:04:04.859611+0000","last_peered":"2024-04-22T07:04:04.859611+0000","last_clean":"2024-04-22T07:04:04.859611+0000","last_became_active":"2024-04-22T07:04:03.789463+0000","last_became_peered":"2024-04-22T07:04:03.789463+0000","last_unstale":"2024-04-22T07:04:04.859611+0000","last_undegraded":"2024-04-22T07:04:04.859611+0000","last_fullsized":"2024-04-22T07:04:04.859611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:16.859332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859989+0000","last_change":"2024-04-22T07:04:02.790600+0000","last_active":"2024-04-22T07:04:04.859989+0000","last_peered":"2024-04-22T07:04:04.859989+0000","last_clean":"2024-04-22T07:04:04.859989+0000","last_became_active":"2024-04-22T07:04:02.790319+0000","last_became_peered":"2024-04-22T07:04:02.790319+0000","last_unstale":"2024-04-22T07:04:04.859989+0000","last_undegraded":"2024-04-22T07:04:04.859989+0000","last_fullsized":"2024-04-22T07:04:04.859989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787840+0000","last_change":"2024-04-22T07:04:03.788231+0000","last_active":"2024-04-22T07:04:04.787840+0000","last_peered":"2024-04-22T07:04:04.787840+0000","last_clean":"2024-04-22T07:04:04.787840+0000","last_became_active":"2024-04-22T07:04:03.787918+0000","last_became_peered":"2024-04-22T07:04:03.787918+0000","last_unstale":"2024-04-22T07:04:04.787840+0000","last_undegraded":"2024-04-22T07:04:04.787840+0000","last_fullsized":"2024-04-22T07:04:04.787840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861257+0000","last_change":"2024-04-22T07:04:02.796245+0000","last_active":"2024-04-22T07:04:04.861257+0000","last_peered":"2024-04-22T07:04:04.861257+0000","last_clean":"2024-04-22T07:04:04.861257+0000","last_became_active":"2024-04-22T07:04:02.795947+0000","last_became_peered":"2024-04-22T07:04:02.795947+0000","last_unstale":"2024-04-22T07:04:04.861257+0000","last_undegraded":"2024-04-22T07:04:04.861257+0000","last_fullsized":"2024-04-22T07:04:04.861257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790274+0000","last_change":"2024-04-22T07:04:03.793330+0000","last_active":"2024-04-22T07:04:04.790274+0000","last_peered":"2024-04-22T07:04:04.790274+0000","last_clean":"2024-04-22T07:04:04.790274+0000","last_became_active":"2024-04-22T07:04:03.793037+0000","last_became_peered":"2024-04-22T07:04:03.793037+0000","last_unstale":"2024-04-22T07:04:04.790274+0000","last_undegraded":"2024-04-22T07:04:04.790274+0000","last_fullsized":"2024-04-22T07:04:04.790274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:13.368853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787950+0000","last_change":"2024-04-22T07:04:02.793498+0000","last_active":"2024-04-22T07:04:04.787950+0000","last_peered":"2024-04-22T07:04:04.787950+0000","last_clean":"2024-04-22T07:04:04.787950+0000","last_became_active":"2024-04-22T07:04:02.793215+0000","last_became_peered":"2024-04-22T07:04:02.793215+0000","last_unstale":"2024-04-22T07:04:04.787950+0000","last_undegraded":"2024-04-22T07:04:04.787950+0000","last_fullsized":"2024-04-22T07:04:04.787950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860866+0000","last_change":"2024-04-22T07:04:03.804152+0000","last_active":"2024-04-22T07:04:04.860866+0000","last_peered":"2024-04-22T07:04:04.860866+0000","last_clean":"2024-04-22T07:04:04.860866+0000","last_became_active":"2024-04-22T07:04:03.801961+0000","last_became_peered":"2024-04-22T07:04:03.801961+0000","last_unstale":"2024-04-22T07:04:04.860866+0000","last_undegraded":"2024-04-22T07:04:04.860866+0000","last_fullsized":"2024-04-22T07:04:04.860866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862105+0000","last_change":"2024-04-22T07:04:02.793445+0000","last_active":"2024-04-22T07:04:04.862105+0000","last_peered":"2024-04-22T07:04:04.862105+0000","last_clean":"2024-04-22T07:04:04.862105+0000","last_became_active":"2024-04-22T07:04:02.793164+0000","last_became_peered":"2024-04-22T07:04:02.793164+0000","last_unstale":"2024-04-22T07:04:04.862105+0000","last_undegraded":"2024-04-22T07:04:04.862105+0000","last_fullsized":"2024-04-22T07:04:04.862105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788013+0000","last_change":"2024-04-22T07:04:03.789589+0000","last_active":"2024-04-22T07:04:04.788013+0000","last_peered":"2024-04-22T07:04:04.788013+0000","last_clean":"2024-04-22T07:04:04.788013+0000","last_became_active":"2024-04-22T07:04:03.789310+0000","last_became_peered":"2024-04-22T07:04:03.789310+0000","last_unstale":"2024-04-22T07:04:04.788013+0000","last_undegraded":"2024-04-22T07:04:04.788013+0000","last_fullsized":"2024-04-22T07:04:04.788013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788077+0000","last_change":"2024-04-22T07:04:02.789243+0000","last_active":"2024-04-22T07:04:04.788077+0000","last_peered":"2024-04-22T07:04:04.788077+0000","last_clean":"2024-04-22T07:04:04.788077+0000","last_became_active":"2024-04-22T07:04:02.788897+0000","last_became_peered":"2024-04-22T07:04:02.788897+0000","last_unstale":"2024-04-22T07:04:04.788077+0000","last_undegraded":"2024-04-22T07:04:04.788077+0000","last_fullsized":"2024-04-22T07:04:04.788077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860272+0000","last_change":"2024-04-22T07:04:03.793656+0000","last_active":"2024-04-22T07:04:04.860272+0000","last_peered":"2024-04-22T07:04:04.860272+0000","last_clean":"2024-04-22T07:04:04.860272+0000","last_became_active":"2024-04-22T07:04:03.793311+0000","last_became_peered":"2024-04-22T07:04:03.793311+0000","last_unstale":"2024-04-22T07:04:04.860272+0000","last_undegraded":"2024-04-22T07:04:04.860272+0000","last_fullsized":"2024-04-22T07:04:04.860272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860713+0000","last_change":"2024-04-22T07:04:02.786878+0000","last_active":"2024-04-22T07:04:04.860713+0000","last_peered":"2024-04-22T07:04:04.860713+0000","last_clean":"2024-04-22T07:04:04.860713+0000","last_became_active":"2024-04-22T07:04:02.786526+0000","last_became_peered":"2024-04-22T07:04:02.786526+0000","last_unstale":"2024-04-22T07:04:04.860713+0000","last_undegraded":"2024-04-22T07:04:04.860713+0000","last_fullsized":"2024-04-22T07:04:04.860713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860520+0000","last_change":"2024-04-22T07:04:03.791544+0000","last_active":"2024-04-22T07:04:04.860520+0000","last_peered":"2024-04-22T07:04:04.860520+0000","last_clean":"2024-04-22T07:04:04.860520+0000","last_became_active":"2024-04-22T07:04:03.791289+0000","last_became_peered":"2024-04-22T07:04:03.791289+0000","last_unstale":"2024-04-22T07:04:04.860520+0000","last_undegraded":"2024-04-22T07:04:04.860520+0000","last_fullsized":"2024-04-22T07:04:04.860520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859963+0000","last_change":"2024-04-22T07:04:02.795178+0000","last_active":"2024-04-22T07:04:04.859963+0000","last_peered":"2024-04-22T07:04:04.859963+0000","last_clean":"2024-04-22T07:04:04.859963+0000","last_became_active":"2024-04-22T07:04:02.794795+0000","last_became_peered":"2024-04-22T07:04:02.794795+0000","last_unstale":"2024-04-22T07:04:04.859963+0000","last_undegraded":"2024-04-22T07:04:04.859963+0000","last_fullsized":"2024-04-22T07:04:04.859963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790163+0000","last_change":"2024-04-22T07:04:03.788560+0000","last_active":"2024-04-22T07:04:04.790163+0000","last_peered":"2024-04-22T07:04:04.790163+0000","last_clean":"2024-04-22T07:04:04.790163+0000","last_became_active":"2024-04-22T07:04:03.788278+0000","last_became_peered":"2024-04-22T07:04:03.788278+0000","last_unstale":"2024-04-22T07:04:04.790163+0000","last_undegraded":"2024-04-22T07:04:04.790163+0000","last_fullsized":"2024-04-22T07:04:04.790163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:37.492038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787876+0000","last_change":"2024-04-22T07:04:02.788532+0000","last_active":"2024-04-22T07:04:04.787876+0000","last_peered":"2024-04-22T07:04:04.787876+0000","last_clean":"2024-04-22T07:04:04.787876+0000","last_became_active":"2024-04-22T07:04:02.788254+0000","last_became_peered":"2024-04-22T07:04:02.788254+0000","last_unstale":"2024-04-22T07:04:04.787876+0000","last_undegraded":"2024-04-22T07:04:04.787876+0000","last_fullsized":"2024-04-22T07:04:04.787876+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859748+0000","last_change":"2024-04-22T07:04:03.790052+0000","last_active":"2024-04-22T07:04:04.859748+0000","last_peered":"2024-04-22T07:04:04.859748+0000","last_clean":"2024-04-22T07:04:04.859748+0000","last_became_active":"2024-04-22T07:04:03.789351+0000","last_became_peered":"2024-04-22T07:04:03.789351+0000","last_unstale":"2024-04-22T07:04:04.859748+0000","last_undegraded":"2024-04-22T07:04:04.859748+0000","last_fullsized":"2024-04-22T07:04:04.859748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787903+0000","last_change":"2024-04-22T07:04:02.792774+0000","last_active":"2024-04-22T07:04:04.787903+0000","last_peered":"2024-04-22T07:04:04.787903+0000","last_clean":"2024-04-22T07:04:04.787903+0000","last_became_active":"2024-04-22T07:04:02.792282+0000","last_became_peered":"2024-04-22T07:04:02.792282+0000","last_unstale":"2024-04-22T07:04:04.787903+0000","last_undegraded":"2024-04-22T07:04:04.787903+0000","last_fullsized":"2024-04-22T07:04:04.787903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860691+0000","last_change":"2024-04-22T07:04:03.796133+0000","last_active":"2024-04-22T07:04:04.860691+0000","last_peered":"2024-04-22T07:04:04.860691+0000","last_clean":"2024-04-22T07:04:04.860691+0000","last_became_active":"2024-04-22T07:04:03.795854+0000","last_became_peered":"2024-04-22T07:04:03.795854+0000","last_unstale":"2024-04-22T07:04:04.860691+0000","last_undegraded":"2024-04-22T07:04:04.860691+0000","last_fullsized":"2024-04-22T07:04:04.860691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861776+0000","last_change":"2024-04-22T07:04:02.789099+0000","last_active":"2024-04-22T07:04:04.861776+0000","last_peered":"2024-04-22T07:04:04.861776+0000","last_clean":"2024-04-22T07:04:04.861776+0000","last_became_active":"2024-04-22T07:04:02.788782+0000","last_became_peered":"2024-04-22T07:04:02.788782+0000","last_unstale":"2024-04-22T07:04:04.861776+0000","last_undegraded":"2024-04-22T07:04:04.861776+0000","last_fullsized":"2024-04-22T07:04:04.861776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860112+0000","last_change":"2024-04-22T07:04:03.799845+0000","last_active":"2024-04-22T07:04:04.860112+0000","last_peered":"2024-04-22T07:04:04.860112+0000","last_clean":"2024-04-22T07:04:04.860112+0000","last_became_active":"2024-04-22T07:04:03.799553+0000","last_became_peered":"2024-04-22T07:04:03.799553+0000","last_unstale":"2024-04-22T07:04:04.860112+0000","last_undegraded":"2024-04-22T07:04:04.860112+0000","last_fullsized":"2024-04-22T07:04:04.860112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860699+0000","last_change":"2024-04-22T07:04:03.799918+0000","last_active":"2024-04-22T07:04:04.860699+0000","last_peered":"2024-04-22T07:04:04.860699+0000","last_clean":"2024-04-22T07:04:04.860699+0000","last_became_active":"2024-04-22T07:04:03.797851+0000","last_became_peered":"2024-04-22T07:04:03.797851+0000","last_unstale":"2024-04-22T07:04:04.860699+0000","last_undegraded":"2024-04-22T07:04:04.860699+0000","last_fullsized":"2024-04-22T07:04:04.860699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861629+0000","last_change":"2024-04-22T07:04:02.793627+0000","last_active":"2024-04-22T07:04:04.861629+0000","last_peered":"2024-04-22T07:04:04.861629+0000","last_clean":"2024-04-22T07:04:04.861629+0000","last_became_active":"2024-04-22T07:04:02.793341+0000","last_became_peered":"2024-04-22T07:04:02.793341+0000","last_unstale":"2024-04-22T07:04:04.861629+0000","last_undegraded":"2024-04-22T07:04:04.861629+0000","last_fullsized":"2024-04-22T07:04:04.861629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:21:07.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861294+0000","last_change":"2024-04-22T07:04:03.797768+0000","last_active":"2024-04-22T07:04:04.861294+0000","last_peered":"2024-04-22T07:04:04.861294+0000","last_clean":"2024-04-22T07:04:04.861294+0000","last_became_active":"2024-04-22T07:04:03.797437+0000","last_became_peered":"2024-04-22T07:04:03.797437+0000","last_unstale":"2024-04-22T07:04:04.861294+0000","last_undegraded":"2024-04-22T07:04:04.861294+0000","last_fullsized":"2024-04-22T07:04:04.861294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861873+0000","last_change":"2024-04-22T07:04:02.789875+0000","last_active":"2024-04-22T07:04:04.861873+0000","last_peered":"2024-04-22T07:04:04.861873+0000","last_clean":"2024-04-22T07:04:04.861873+0000","last_became_active":"2024-04-22T07:04:02.788989+0000","last_became_peered":"2024-04-22T07:04:02.788989+0000","last_unstale":"2024-04-22T07:04:04.861873+0000","last_undegraded":"2024-04-22T07:04:04.861873+0000","last_fullsized":"2024-04-22T07:04:04.861873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860096+0000","last_change":"2024-04-22T07:04:03.797671+0000","last_active":"2024-04-22T07:04:04.860096+0000","last_peered":"2024-04-22T07:04:04.860096+0000","last_clean":"2024-04-22T07:04:04.860096+0000","last_became_active":"2024-04-22T07:04:03.797399+0000","last_became_peered":"2024-04-22T07:04:03.797399+0000","last_unstale":"2024-04-22T07:04:04.860096+0000","last_undegraded":"2024-04-22T07:04:04.860096+0000","last_fullsized":"2024-04-22T07:04:04.860096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860425+0000","last_change":"2024-04-22T07:04:02.796141+0000","last_active":"2024-04-22T07:04:04.860425+0000","last_peered":"2024-04-22T07:04:04.860425+0000","last_clean":"2024-04-22T07:04:04.860425+0000","last_became_active":"2024-04-22T07:04:02.795821+0000","last_became_peered":"2024-04-22T07:04:02.795821+0000","last_unstale":"2024-04-22T07:04:04.860425+0000","last_undegraded":"2024-04-22T07:04:04.860425+0000","last_fullsized":"2024-04-22T07:04:04.860425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860371+0000","last_change":"2024-04-22T07:04:03.796729+0000","last_active":"2024-04-22T07:04:04.860371+0000","last_peered":"2024-04-22T07:04:04.860371+0000","last_clean":"2024-04-22T07:04:04.860371+0000","last_became_active":"2024-04-22T07:04:03.796486+0000","last_became_peered":"2024-04-22T07:04:03.796486+0000","last_unstale":"2024-04-22T07:04:04.860371+0000","last_undegraded":"2024-04-22T07:04:04.860371+0000","last_fullsized":"2024-04-22T07:04:04.860371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860322+0000","last_change":"2024-04-22T07:04:02.793214+0000","last_active":"2024-04-22T07:04:04.860322+0000","last_peered":"2024-04-22T07:04:04.860322+0000","last_clean":"2024-04-22T07:04:04.860322+0000","last_became_active":"2024-04-22T07:04:02.792922+0000","last_became_peered":"2024-04-22T07:04:02.792922+0000","last_unstale":"2024-04-22T07:04:04.860322+0000","last_undegraded":"2024-04-22T07:04:04.860322+0000","last_fullsized":"2024-04-22T07:04:04.860322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":661302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":661302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:09.092 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-22T07:04:09.092 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-22T07:04:09.093 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-22T07:04:09.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-22T07:04:09.093 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-22T07:04:09.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.226+0000 7fbe8756d640 1 Processor -- start 2024-04-22T07:04:09.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.226+0000 7fbe8756d640 1 -- start start 2024-04-22T07:04:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.227+0000 7fbe8756d640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fbe8006c7d0 0x7fbe80154a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.227+0000 7fbe8756d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe8015eeb0 con 0x7fbe8006c3f0 2024-04-22T07:04:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.227+0000 7fbe8756d640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe800c9f60 con 0x7fbe8006b970 2024-04-22T07:04:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.227+0000 7fbe8756d640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fbe8006fad0 con 0x7fbe8006c7d0 2024-04-22T07:04:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.227+0000 7fbe85309640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbe8006c3f0 0x7fbe8006ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46584/0 (socket says 172.21.15.92:46584) 2024-04-22T07:04:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.227+0000 7fbe85309640 1 -- 172.21.15.92:0/1428185337 learned_addr learned my addr 172.21.15.92:0/1428185337 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe84b08640 1 --2- 172.21.15.92:0/1428185337 >> v2:172.21.15.145:3300/0 conn(0x7fbe8006c7d0 0x7fbe80154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1487493743 0 0) 0x7fbe8015eeb0 con 0x7fbe8006c3f0 2024-04-22T07:04:09.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe6c003520 con 0x7fbe8006c3f0 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3107124616 0 0) 0x7fbe6c003520 con 0x7fbe8006c3f0 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 >> v2:172.21.15.145:3300/0 conn(0x7fbe8006c7d0 msgr2=0x7fbe80154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 --2- 172.21.15.92:0/1428185337 >> v2:172.21.15.145:3300/0 conn(0x7fbe8006c7d0 0x7fbe80154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 >> v1:172.21.15.145:6789/0 conn(0x7fbe8006b970 legacy=0x7fbe801654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe800ca240 con 0x7fbe8006c3f0 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7fbe7c0019f0 con 0x7fbe8006c3f0 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 >> v1:172.21.15.92:6789/0 conn(0x7fbe8006c3f0 legacy=0x7fbe8006ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 --2- 172.21.15.92:0/1428185337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe6c0039b0 0x7fbe6c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 --2- 172.21.15.92:0/1428185337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe6c004800 0x7fbe6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 --2- 172.21.15.92:0/1428185337 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe6c0051c0 0x7fbe6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8006fad0 con 0x7fbe6c0039b0 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c00c090 con 0x7fbe6c004800 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6c00c290 con 0x7fbe6c0051c0 2024-04-22T07:04:09.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe85309640 1 --2- 172.21.15.92:0/1428185337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe6c0039b0 0x7fbe6c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe84b08640 1 --2- 172.21.15.92:0/1428185337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe6c004800 0x7fbe6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe85b0a640 1 --2- 172.21.15.92:0/1428185337 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe6c0051c0 0x7fbe6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe85309640 1 -- 172.21.15.92:0/1428185337 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe6c0051c0 msgr2=0x7fbe6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe85309640 1 --2- 172.21.15.92:0/1428185337 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe6c0051c0 0x7fbe6c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe85309640 1 -- 172.21.15.92:0/1428185337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe6c004800 msgr2=0x7fbe6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe85309640 1 --2- 172.21.15.92:0/1428185337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe6c004800 0x7fbe6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe85309640 1 -- 172.21.15.92:0/1428185337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c003520 con 0x7fbe6c0039b0 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.228+0000 7fbe85b0a640 1 -- 172.21.15.92:0/1428185337 reap_dead start 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.229+0000 7fbe85309640 1 --2- 172.21.15.92:0/1428185337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe6c0039b0 0x7fbe6c003df0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00a840 tx=0x7fbe6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c422edfe91e7afa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.229+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00a460 con 0x7fbe6c0039b0 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.229+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe7c034650 con 0x7fbe6c0039b0 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.229+0000 7fbe77fff640 1 -- 172.21.15.92:0/1428185337 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c034a40 con 0x7fbe6c0039b0 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.230+0000 7fbe8756d640 1 -- 172.21.15.92:0/1428185337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe6c0039b0 msgr2=0x7fbe6c003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.230+0000 7fbe8756d640 1 --2- 172.21.15.92:0/1428185337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe6c0039b0 0x7fbe6c003df0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00a840 tx=0x7fbe6c005830 comp rx=0 tx=0).stop 2024-04-22T07:04:09.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.230+0000 7fbe8756d640 1 -- 172.21.15.92:0/1428185337 shutdown_connections 2024-04-22T07:04:09.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.230+0000 7fbe8756d640 1 --2- 172.21.15.92:0/1428185337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe6c0039b0 0x7fbe6c003df0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.230+0000 7fbe8756d640 1 -- 172.21.15.92:0/1428185337 >> 172.21.15.92:0/1428185337 conn(0x7fbe8006e260 msgr2=0x7fbe800715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:09.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.230+0000 7fbe8756d640 1 -- 172.21.15.92:0/1428185337 shutdown_connections 2024-04-22T07:04:09.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.230+0000 7fbe8756d640 1 -- 172.21.15.92:0/1428185337 wait complete. 2024-04-22T07:04:09.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.231+0000 7fbe8756d640 1 Processor -- start 2024-04-22T07:04:09.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.231+0000 7fbe8756d640 1 -- start start 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.231+0000 7fbe8756d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe6c0039b0 0x7fbe801f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.231+0000 7fbe8756d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe80147770 0x7fbe80150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.231+0000 7fbe8756d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe801507a0 0x7fbe80153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe8756d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbe800c8f70 con 0x7fbe80147770 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe8756d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbe800c8df0 con 0x7fbe6c0039b0 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe8756d640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbe800720e0 con 0x7fbe801507a0 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe84b08640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe80147770 0x7fbe80150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe84b08640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe80147770 0x7fbe80150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35998/0 (socket says 172.21.15.92:35998) 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe84b08640 1 -- 172.21.15.92:0/3815406818 learned_addr learned my addr 172.21.15.92:0/3815406818 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe85b0a640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe801507a0 0x7fbe80153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe85309640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe6c0039b0 0x7fbe801f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe84b08640 1 -- 172.21.15.92:0/3815406818 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe801507a0 msgr2=0x7fbe80153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe84b08640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe801507a0 0x7fbe80153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe84b08640 1 -- 172.21.15.92:0/3815406818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe6c0039b0 msgr2=0x7fbe801f0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe84b08640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe6c0039b0 0x7fbe801f0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe84b08640 1 -- 172.21.15.92:0/3815406818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8014e0d0 con 0x7fbe80147770 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe85309640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe6c0039b0 0x7fbe801f0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe84b08640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe80147770 0x7fbe80150260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe70011640 tx=0x7fbe70011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe75ffb640 1 -- 172.21.15.92:0/3815406818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70004220 con 0x7fbe80147770 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe75ffb640 1 -- 172.21.15.92:0/3815406818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe700043c0 con 0x7fbe80147770 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe75ffb640 1 -- 172.21.15.92:0/3815406818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe700158b0 con 0x7fbe80147770 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe8756d640 1 -- 172.21.15.92:0/3815406818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8014fb90 con 0x7fbe80147770 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.232+0000 7fbe8756d640 1 -- 172.21.15.92:0/3815406818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8014e350 con 0x7fbe80147770 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.233+0000 7fbe8756d640 1 -- 172.21.15.92:0/3815406818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe80068b00 con 0x7fbe80147770 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.234+0000 7fbe75ffb640 1 -- 172.21.15.92:0/3815406818 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe70015a50 con 0x7fbe80147770 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.234+0000 7fbe75ffb640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fbe6406c8f0 0x7fbe6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.234+0000 7fbe85309640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fbe6406c8f0 0x7fbe6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.234+0000 7fbe75ffb640 1 -- 172.21.15.92:0/3815406818 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe7008bbe0 con 0x7fbe80147770 2024-04-22T07:04:09.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.235+0000 7fbe85309640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fbe6406c8f0 0x7fbe6406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00a840 tx=0x7fbe7c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:09.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.239+0000 7fbe75ffb640 1 -- 172.21.15.92:0/3815406818 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe8014e0d0 con 0x7fbe80147770 2024-04-22T07:04:09.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.460+0000 7fbe8756d640 1 -- 172.21.15.92:0/3815406818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fbe8006cff0 con 0x7fbe80147770 2024-04-22T07:04:09.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.463+0000 7fbe75ffb640 1 -- 172.21.15.92:0/3815406818 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174133 (secure 0 0 0) 0x7fbe70019030 con 0x7fbe80147770 2024-04-22T07:04:09.463 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:09.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.466+0000 7fbe8756d640 1 -- 172.21.15.92:0/3815406818 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fbe6406c8f0 msgr2=0x7fbe6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.466+0000 7fbe8756d640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fbe6406c8f0 0x7fbe6406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00a840 tx=0x7fbe7c009030 comp rx=0 tx=0).stop 2024-04-22T07:04:09.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.466+0000 7fbe8756d640 1 -- 172.21.15.92:0/3815406818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe80147770 msgr2=0x7fbe80150260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.466+0000 7fbe8756d640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe80147770 0x7fbe80150260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe70011640 tx=0x7fbe70011b00 comp rx=0 tx=0).stop 2024-04-22T07:04:09.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.466+0000 7fbe8756d640 1 -- 172.21.15.92:0/3815406818 shutdown_connections 2024-04-22T07:04:09.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.466+0000 7fbe8756d640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fbe6406c8f0 0x7fbe6406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.466+0000 7fbe8756d640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe801507a0 0x7fbe80153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.466+0000 7fbe8756d640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe80147770 0x7fbe80150260 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.466+0000 7fbe8756d640 1 --2- 172.21.15.92:0/3815406818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe6c0039b0 0x7fbe801f0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.466+0000 7fbe8756d640 1 -- 172.21.15.92:0/3815406818 >> 172.21.15.92:0/3815406818 conn(0x7fbe8006e260 msgr2=0x7fbe80156170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:09.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.466+0000 7fbe8756d640 1 -- 172.21.15.92:0/3815406818 shutdown_connections 2024-04-22T07:04:09.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.466+0000 7fbe8756d640 1 -- 172.21.15.92:0/3815406818 wait complete. 2024-04-22T07:04:09.474 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":479317737},{"type":"v1","addr":"172.21.15.92:6833","nonce":479317737}]},"active_addr":"172.21.15.92:6833/479317737","active_change":"2024-04-22T07:03:49.926378+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":1278638567}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3480454667}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3311569901}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":105315326}]}]} 2024-04-22T07:04:09.478 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-22T07:04:09.478 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-22T07:04:09.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:09.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.577+0000 7f8244732640 1 Processor -- start 2024-04-22T07:04:09.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.577+0000 7f8244732640 1 -- start start 2024-04-22T07:04:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f8244732640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f823c0cd200 0x7f823c15b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f8244732640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f823c0cd5e0 0x7f823c154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f8244732640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f823c15e660 con 0x7f823c0cb6b0 2024-04-22T07:04:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f8244732640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f823c15f480 con 0x7f823c0cd200 2024-04-22T07:04:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f8244732640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f823c15e900 con 0x7f823c0cd5e0 2024-04-22T07:04:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f82424ce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f823c0cd200 0x7f823c15b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f82424ce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f823c0cd200 0x7f823c15b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36014/0 (socket says 172.21.15.92:36014) 2024-04-22T07:04:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f82424ce640 1 -- 172.21.15.92:0/535954733 learned_addr learned my addr 172.21.15.92:0/535954733 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f8241ccd640 1 --2- 172.21.15.92:0/535954733 >> v2:172.21.15.145:3300/0 conn(0x7f823c0cd5e0 0x7f823c154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f82424ce640 1 -- 172.21.15.92:0/535954733 >> v2:172.21.15.145:3300/0 conn(0x7f823c0cd5e0 msgr2=0x7f823c154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f82424ce640 1 --2- 172.21.15.92:0/535954733 >> v2:172.21.15.145:3300/0 conn(0x7f823c0cd5e0 0x7f823c154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f82424ce640 1 -- 172.21.15.92:0/535954733 >> v1:172.21.15.145:6789/0 conn(0x7f823c0cb6b0 legacy=0x7f823c0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f82424ce640 1 -- 172.21.15.92:0/535954733 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c15ebe0 con 0x7f823c0cd200 2024-04-22T07:04:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.578+0000 7f8241ccd640 1 --2- 172.21.15.92:0/535954733 >> v2:172.21.15.145:3300/0 conn(0x7f823c0cd5e0 0x7f823c154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.579+0000 7f82424ce640 1 --2- 172.21.15.92:0/535954733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c0cd200 0x7f823c15b9b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f823c0741b0 tx=0x7f823802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5201fcd738f6b783 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.579+0000 7f82414cc640 1 -- 172.21.15.92:0/535954733 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238034070 con 0x7f823c0cd200 2024-04-22T07:04:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.579+0000 7f82414cc640 1 -- 172.21.15.92:0/535954733 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8238030860 con 0x7f823c0cd200 2024-04-22T07:04:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.579+0000 7f82414cc640 1 -- 172.21.15.92:0/535954733 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238030c30 con 0x7f823c0cd200 2024-04-22T07:04:09.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.579+0000 7f8244732640 1 -- 172.21.15.92:0/535954733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c0cd200 msgr2=0x7f823c15b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.579+0000 7f8244732640 1 --2- 172.21.15.92:0/535954733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c0cd200 0x7f823c15b9b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f823c0741b0 tx=0x7f823802b9a0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.579+0000 7f8244732640 1 -- 172.21.15.92:0/535954733 shutdown_connections 2024-04-22T07:04:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.579+0000 7f8244732640 1 --2- 172.21.15.92:0/535954733 >> v2:172.21.15.145:3300/0 conn(0x7f823c0cd5e0 0x7f823c154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.579+0000 7f8244732640 1 --2- 172.21.15.92:0/535954733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c0cd200 0x7f823c15b9b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.579+0000 7f8244732640 1 -- 172.21.15.92:0/535954733 >> 172.21.15.92:0/535954733 conn(0x7f823c073bc0 msgr2=0x7f823c0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.580+0000 7f8244732640 1 -- 172.21.15.92:0/535954733 shutdown_connections 2024-04-22T07:04:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.580+0000 7f8244732640 1 -- 172.21.15.92:0/535954733 wait complete. 2024-04-22T07:04:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.581+0000 7f8244732640 1 Processor -- start 2024-04-22T07:04:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.581+0000 7f8244732640 1 -- start start 2024-04-22T07:04:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.581+0000 7f8244732640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c0cb6b0 0x7f823c1507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.581+0000 7f8244732640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f823c0cd200 0x7f823c0c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.581+0000 7f8244732640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f823c0cd5e0 0x7f823c1f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.581+0000 7f8244732640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f823c0cc430 con 0x7f823c0cb6b0 2024-04-22T07:04:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.581+0000 7f8244732640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f823c15f1a0 con 0x7f823c0cd200 2024-04-22T07:04:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.581+0000 7f8244732640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f823c0cc5b0 con 0x7f823c0cd5e0 2024-04-22T07:04:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.581+0000 7f82424ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c0cb6b0 0x7f823c1507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.581+0000 7f82424ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c0cb6b0 0x7f823c1507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36018/0 (socket says 172.21.15.92:36018) 2024-04-22T07:04:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.581+0000 7f82424ce640 1 -- 172.21.15.92:0/1954145939 learned_addr learned my addr 172.21.15.92:0/1954145939 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.582+0000 7f82424ce640 1 -- 172.21.15.92:0/1954145939 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f823c0cd5e0 msgr2=0x7f823c1f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.582+0000 7f82424ce640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f823c0cd5e0 0x7f823c1f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.582+0000 7f82424ce640 1 -- 172.21.15.92:0/1954145939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f823c0cd200 msgr2=0x7f823c0c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.582+0000 7f8241ccd640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f823c0cd200 0x7f823c0c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.582+0000 7f82424ce640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f823c0cd200 0x7f823c0c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.582+0000 7f82424ce640 1 -- 172.21.15.92:0/1954145939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c0c9ed0 con 0x7f823c0cb6b0 2024-04-22T07:04:09.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.582+0000 7f82424ce640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c0cb6b0 0x7f823c1507e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8238000c00 tx=0x7f8238004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:09.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.582+0000 7f8232ffd640 1 -- 172.21.15.92:0/1954145939 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238048070 con 0x7f823c0cb6b0 2024-04-22T07:04:09.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.582+0000 7f8232ffd640 1 -- 172.21.15.92:0/1954145939 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8238039d90 con 0x7f823c0cb6b0 2024-04-22T07:04:09.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.582+0000 7f8244732640 1 -- 172.21.15.92:0/1954145939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823c151670 con 0x7f823c0cb6b0 2024-04-22T07:04:09.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.582+0000 7f8232ffd640 1 -- 172.21.15.92:0/1954145939 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238034040 con 0x7f823c0cb6b0 2024-04-22T07:04:09.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.582+0000 7f8244732640 1 -- 172.21.15.92:0/1954145939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823c0ca420 con 0x7f823c0cb6b0 2024-04-22T07:04:09.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.583+0000 7f8244732640 1 -- 172.21.15.92:0/1954145939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f823c06c800 con 0x7f823c0cb6b0 2024-04-22T07:04:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.583+0000 7f8232ffd640 1 -- 172.21.15.92:0/1954145939 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f823804b070 con 0x7f823c0cb6b0 2024-04-22T07:04:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.584+0000 7f8232ffd640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f822006c920 0x7f822006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.584+0000 7f8232ffd640 1 -- 172.21.15.92:0/1954145939 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f82380adc30 con 0x7f823c0cb6b0 2024-04-22T07:04:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.584+0000 7f8241ccd640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f822006c920 0x7f822006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.584+0000 7f8241ccd640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f822006c920 0x7f822006ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f823c153c80 tx=0x7f822c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:09.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.588+0000 7f8232ffd640 1 -- 172.21.15.92:0/1954145939 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f823c06c800 con 0x7f823c0cb6b0 2024-04-22T07:04:09.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.765+0000 7f8244732640 1 -- 172.21.15.92:0/1954145939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f823c0ca990 con 0x7f823c0cb6b0 2024-04-22T07:04:09.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.766+0000 7f8232ffd640 1 -- 172.21.15.92:0/1954145939 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f823c151670 con 0x7f823c0cb6b0 2024-04-22T07:04:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:09.764 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:04:04.784969+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:09.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.768+0000 7f8244732640 1 -- 172.21.15.92:0/1954145939 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f822006c920 msgr2=0x7f822006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.768+0000 7f8244732640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f822006c920 0x7f822006ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f823c153c80 tx=0x7f822c008040 comp rx=0 tx=0).stop 2024-04-22T07:04:09.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.768+0000 7f8244732640 1 -- 172.21.15.92:0/1954145939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c0cb6b0 msgr2=0x7f823c1507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.768+0000 7f8244732640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c0cb6b0 0x7f823c1507e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8238000c00 tx=0x7f8238004730 comp rx=0 tx=0).stop 2024-04-22T07:04:09.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.768+0000 7f8244732640 1 -- 172.21.15.92:0/1954145939 shutdown_connections 2024-04-22T07:04:09.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.768+0000 7f8244732640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f822006c920 0x7f822006ede0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.768+0000 7f8244732640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f823c0cd5e0 0x7f823c1f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.768+0000 7f8244732640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f823c0cd200 0x7f823c0c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.768+0000 7f8244732640 1 --2- 172.21.15.92:0/1954145939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823c0cb6b0 0x7f823c1507e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.768+0000 7f8244732640 1 -- 172.21.15.92:0/1954145939 >> 172.21.15.92:0/1954145939 conn(0x7f823c073bc0 msgr2=0x7f823c0c54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:09.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.768+0000 7f8244732640 1 -- 172.21.15.92:0/1954145939 shutdown_connections 2024-04-22T07:04:09.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.768+0000 7f8244732640 1 -- 172.21.15.92:0/1954145939 wait complete. 2024-04-22T07:04:09.777 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-22T07:04:09.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:09.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.876+0000 7f1c777ce640 1 Processor -- start 2024-04-22T07:04:09.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.876+0000 7f1c777ce640 1 -- start start 2024-04-22T07:04:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c777ce640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1c7015b4a0 0x7f1c7006f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c777ce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1c700c9db0 0x7f1c700ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c777ce640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c700c8e00 con 0x7f1c700ce2f0 2024-04-22T07:04:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c777ce640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f1c700c9a10 con 0x7f1c700c9db0 2024-04-22T07:04:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c777ce640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1c700c90e0 con 0x7f1c7015b4a0 2024-04-22T07:04:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c75d6b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1c700c9db0 0x7f1c700ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c7556a640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1c700ce2f0 0x7f1c7015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46590/0 (socket says 172.21.15.92:46590) 2024-04-22T07:04:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c75d6b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1c700c9db0 0x7f1c700ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36032/0 (socket says 172.21.15.92:36032) 2024-04-22T07:04:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c7556a640 1 -- 172.21.15.92:0/3390236759 learned_addr learned my addr 172.21.15.92:0/3390236759 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:09.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c74d69640 1 --2- 172.21.15.92:0/3390236759 >> v2:172.21.15.145:3300/0 conn(0x7f1c7015b4a0 0x7f1c7006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c75d6b640 1 -- 172.21.15.92:0/3390236759 >> v2:172.21.15.145:3300/0 conn(0x7f1c7015b4a0 msgr2=0x7f1c7006f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c75d6b640 1 --2- 172.21.15.92:0/3390236759 >> v2:172.21.15.145:3300/0 conn(0x7f1c7015b4a0 0x7f1c7006f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c75d6b640 1 -- 172.21.15.92:0/3390236759 >> v1:172.21.15.92:6789/0 conn(0x7f1c700ce2f0 legacy=0x7f1c7015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c75d6b640 1 -- 172.21.15.92:0/3390236759 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c700c93c0 con 0x7f1c700c9db0 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c67fff640 1 -- 172.21.15.92:0/3390236759 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1219565916 0 0) 0x7f1c700c8e00 con 0x7f1c700ce2f0 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.877+0000 7f1c74d69640 1 --2- 172.21.15.92:0/3390236759 >> v2:172.21.15.145:3300/0 conn(0x7f1c7015b4a0 0x7f1c7006f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.878+0000 7f1c75d6b640 1 --2- 172.21.15.92:0/3390236759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c700c9db0 0x7f1c700ce6d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1c7006f4a0 tx=0x7f1c6802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232ec167187c4429 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.878+0000 7f1c67fff640 1 -- 172.21.15.92:0/3390236759 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68039070 con 0x7f1c700c9db0 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.878+0000 7f1c67fff640 1 -- 172.21.15.92:0/3390236759 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6802cd00 con 0x7f1c700c9db0 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.878+0000 7f1c67fff640 1 -- 172.21.15.92:0/3390236759 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68034660 con 0x7f1c700c9db0 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.878+0000 7f1c777ce640 1 -- 172.21.15.92:0/3390236759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c700c9db0 msgr2=0x7f1c700ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.878+0000 7f1c777ce640 1 --2- 172.21.15.92:0/3390236759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c700c9db0 0x7f1c700ce6d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1c7006f4a0 tx=0x7f1c6802bf50 comp rx=0 tx=0).stop 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.878+0000 7f1c777ce640 1 -- 172.21.15.92:0/3390236759 shutdown_connections 2024-04-22T07:04:09.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.878+0000 7f1c777ce640 1 --2- 172.21.15.92:0/3390236759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c700c9db0 0x7f1c700ce6d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.878+0000 7f1c777ce640 1 --2- 172.21.15.92:0/3390236759 >> v2:172.21.15.145:3300/0 conn(0x7f1c7015b4a0 0x7f1c7006f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.878+0000 7f1c777ce640 1 -- 172.21.15.92:0/3390236759 >> 172.21.15.92:0/3390236759 conn(0x7f1c7006db80 msgr2=0x7f1c7006df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:09.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.878+0000 7f1c777ce640 1 -- 172.21.15.92:0/3390236759 shutdown_connections 2024-04-22T07:04:09.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.879+0000 7f1c777ce640 1 -- 172.21.15.92:0/3390236759 wait complete. 2024-04-22T07:04:09.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c777ce640 1 Processor -- start 2024-04-22T07:04:09.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c777ce640 1 -- start start 2024-04-22T07:04:09.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c777ce640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c700c9db0 0x7f1c70154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c777ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c700ce2f0 0x7f1c701f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c777ce640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1c7015b4a0 0x7f1c70155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c777ce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1c70070010 con 0x7f1c700ce2f0 2024-04-22T07:04:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c777ce640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7006d910 con 0x7f1c700c9db0 2024-04-22T07:04:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c777ce640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1c700cbad0 con 0x7f1c7015b4a0 2024-04-22T07:04:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c74d69640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c700ce2f0 0x7f1c701f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c74d69640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c700ce2f0 0x7f1c701f11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36036/0 (socket says 172.21.15.92:36036) 2024-04-22T07:04:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c74d69640 1 -- 172.21.15.92:0/1243863177 learned_addr learned my addr 172.21.15.92:0/1243863177 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c7556a640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c700c9db0 0x7f1c70154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.880+0000 7f1c75d6b640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1c7015b4a0 0x7f1c70155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.881+0000 7f1c74d69640 1 -- 172.21.15.92:0/1243863177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1c7015b4a0 msgr2=0x7f1c70155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.881+0000 7f1c74d69640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1c7015b4a0 0x7f1c70155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.881+0000 7f1c74d69640 1 -- 172.21.15.92:0/1243863177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c700c9db0 msgr2=0x7f1c70154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.881+0000 7f1c74d69640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c700c9db0 0x7f1c70154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.881+0000 7f1c74d69640 1 -- 172.21.15.92:0/1243863177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c701566a0 con 0x7f1c700ce2f0 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.881+0000 7f1c74d69640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c700ce2f0 0x7f1c701f11c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c7f0 tx=0x7f1c6000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.881+0000 7f1c65ffb640 1 -- 172.21.15.92:0/1243863177 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60015070 con 0x7f1c700ce2f0 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.881+0000 7f1c65ffb640 1 -- 172.21.15.92:0/1243863177 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c60007a30 con 0x7f1c700ce2f0 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.881+0000 7f1c777ce640 1 -- 172.21.15.92:0/1243863177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c70156980 con 0x7f1c700ce2f0 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.881+0000 7f1c65ffb640 1 -- 172.21.15.92:0/1243863177 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60007e00 con 0x7f1c700ce2f0 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.881+0000 7f1c777ce640 1 -- 172.21.15.92:0/1243863177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c70154f50 con 0x7f1c700ce2f0 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.882+0000 7f1c777ce640 1 -- 172.21.15.92:0/1243863177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c70069930 con 0x7f1c700ce2f0 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.882+0000 7f1c65ffb640 1 -- 172.21.15.92:0/1243863177 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c60018070 con 0x7f1c700ce2f0 2024-04-22T07:04:09.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.883+0000 7f1c65ffb640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1c5406c8f0 0x7f1c5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:09.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.883+0000 7f1c65ffb640 1 -- 172.21.15.92:0/1243863177 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1c6008ba80 con 0x7f1c700ce2f0 2024-04-22T07:04:09.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.883+0000 7f1c7556a640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1c5406c8f0 0x7f1c5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:09.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.883+0000 7f1c7556a640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1c5406c8f0 0x7f1c5406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c000f60 tx=0x7f1c6c007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:09.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:09.887+0000 7f1c65ffb640 1 -- 172.21.15.92:0/1243863177 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c60059860 con 0x7f1c700ce2f0 2024-04-22T07:04:10.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.062+0000 7f1c777ce640 1 -- 172.21.15.92:0/1243863177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c7006d0a0 con 0x7f1c700ce2f0 2024-04-22T07:04:10.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.063+0000 7f1c65ffb640 1 -- 172.21.15.92:0/1243863177 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f1c6005db60 con 0x7f1c700ce2f0 2024-04-22T07:04:10.062 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:10.062 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:04:04.784969+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:10.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.065+0000 7f1c777ce640 1 -- 172.21.15.92:0/1243863177 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1c5406c8f0 msgr2=0x7f1c5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.065+0000 7f1c777ce640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1c5406c8f0 0x7f1c5406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c000f60 tx=0x7f1c6c007d50 comp rx=0 tx=0).stop 2024-04-22T07:04:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.065+0000 7f1c777ce640 1 -- 172.21.15.92:0/1243863177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c700ce2f0 msgr2=0x7f1c701f11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.065+0000 7f1c777ce640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c700ce2f0 0x7f1c701f11c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c7f0 tx=0x7f1c6000cbf0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.065+0000 7f1c777ce640 1 -- 172.21.15.92:0/1243863177 shutdown_connections 2024-04-22T07:04:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.065+0000 7f1c777ce640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1c5406c8f0 0x7f1c5406edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.065+0000 7f1c777ce640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1c7015b4a0 0x7f1c70155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.065+0000 7f1c777ce640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c700ce2f0 0x7f1c701f11c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.065+0000 7f1c777ce640 1 --2- 172.21.15.92:0/1243863177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c700c9db0 0x7f1c70154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.065+0000 7f1c777ce640 1 -- 172.21.15.92:0/1243863177 >> 172.21.15.92:0/1243863177 conn(0x7f1c7006db80 msgr2=0x7f1c700cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.065+0000 7f1c777ce640 1 -- 172.21.15.92:0/1243863177 shutdown_connections 2024-04-22T07:04:10.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.066+0000 7f1c777ce640 1 -- 172.21.15.92:0/1243863177 wait complete. 2024-04-22T07:04:10.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-22T07:04:10.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-22T07:04:10.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-22T07:04:10.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-22T07:04:10.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-22T07:04:10.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-22T07:04:10.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-22T07:04:10.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-22T07:04:10.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.239+0000 7f309cd4a640 1 Processor -- start 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.239+0000 7f309cd4a640 1 -- start start 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.240+0000 7f309cd4a640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f30980aeb60 0x7f30980c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.240+0000 7f309cd4a640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f309806e430 0x7f309816ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.240+0000 7f309cd4a640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30980c6c10 con 0x7f3098073c10 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.240+0000 7f309cd4a640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f30980c6ef0 con 0x7f309806e430 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.240+0000 7f309cd4a640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f30980c71d0 con 0x7f30980aeb60 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.241+0000 7f309659c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f30980aeb60 0x7f30980c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.241+0000 7f309659c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f30980aeb60 0x7f30980c1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52218/0 (socket says 172.21.15.92:52218) 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.241+0000 7f309659c640 1 -- 172.21.15.92:0/2243147083 learned_addr learned my addr 172.21.15.92:0/2243147083 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.241+0000 7f3096d9d640 1 --2- 172.21.15.92:0/2243147083 >> v2:172.21.15.145:3300/0 conn(0x7f309806e430 0x7f309816ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.241+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3691738349 0 0) 0x7f30980c6c10 con 0x7f3098073c10 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.241+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3080007930 con 0x7f3098073c10 2024-04-22T07:04:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 <== mon.1 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2863682881 0 0) 0x7f3080007930 con 0x7f3098073c10 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f30980aeb60 msgr2=0x7f30980c1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 --2- 172.21.15.92:0/2243147083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f30980aeb60 0x7f30980c1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 >> v2:172.21.15.145:3300/0 conn(0x7f309806e430 msgr2=0x7f309816ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 --2- 172.21.15.92:0/2243147083 >> v2:172.21.15.145:3300/0 conn(0x7f309806e430 0x7f309816ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309806d520 con 0x7f3098073c10 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 <== mon.1 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f3088001670 con 0x7f3098073c10 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 >> v1:172.21.15.145:6789/0 conn(0x7f3098073c10 legacy=0x7f309806dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 --2- 172.21.15.92:0/2243147083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3080007dc0 0x7f30800081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 --2- 172.21.15.92:0/2243147083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3080008c10 0x7f3080009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 --2- 172.21.15.92:0/2243147083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3080007440 0x7f30800095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30980c6ef0 con 0x7f3080008c10 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f30980c71d0 con 0x7f3080007dc0 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f308001cea0 con 0x7f3080007440 2024-04-22T07:04:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f3095d9b640 1 --2- 172.21.15.92:0/2243147083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3080008c10 0x7f3080009090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f309659c640 1 --2- 172.21.15.92:0/2243147083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3080007dc0 0x7f30800081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f3096d9d640 1 --2- 172.21.15.92:0/2243147083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3080007440 0x7f30800095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f3095d9b640 1 -- 172.21.15.92:0/2243147083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3080007440 msgr2=0x7f30800095d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f3095d9b640 1 --2- 172.21.15.92:0/2243147083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3080007440 0x7f30800095d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.242+0000 7f3095d9b640 1 -- 172.21.15.92:0/2243147083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3080007dc0 msgr2=0x7f30800081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.243+0000 7f3095d9b640 1 --2- 172.21.15.92:0/2243147083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3080007dc0 0x7f30800081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.243+0000 7f3095d9b640 1 -- 172.21.15.92:0/2243147083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309806d520 con 0x7f3080008c10 2024-04-22T07:04:10.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.244+0000 7f3095d9b640 1 --2- 172.21.15.92:0/2243147083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3080008c10 0x7f3080009090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f308800c1c0 tx=0x7f3080009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff3f10110d9de8d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.244+0000 7fc32f396640 1 Processor -- start 2024-04-22T07:04:10.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.244+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088039070 con 0x7f3080008c10 2024-04-22T07:04:10.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.244+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308802ca70 con 0x7f3080008c10 2024-04-22T07:04:10.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.244+0000 7f309559a640 1 -- 172.21.15.92:0/2243147083 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088007c80 con 0x7f3080008c10 2024-04-22T07:04:10.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.244+0000 7f222f1ca640 1 Processor -- start 2024-04-22T07:04:10.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.244+0000 7f222f1ca640 1 -- start start 2024-04-22T07:04:10.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.245+0000 7ff43109a640 1 Processor -- start 2024-04-22T07:04:10.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.245+0000 7f309cd4a640 1 -- 172.21.15.92:0/2243147083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3080008c10 msgr2=0x7f3080009090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.245+0000 7f309cd4a640 1 --2- 172.21.15.92:0/2243147083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3080008c10 0x7f3080009090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f308800c1c0 tx=0x7f3080009c40 comp rx=0 tx=0).stop 2024-04-22T07:04:10.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.245+0000 7f3096d9d640 1 -- 172.21.15.92:0/2243147083 reap_dead start 2024-04-22T07:04:10.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.245+0000 7ff43109a640 1 -- start start 2024-04-22T07:04:10.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.245+0000 7fc32f396640 1 -- start start 2024-04-22T07:04:10.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7f2da6c06640 1 Processor -- start 2024-04-22T07:04:10.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7fc32f396640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3200db210 0x7fc320006ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7f309cd4a640 1 -- 172.21.15.92:0/2243147083 shutdown_connections 2024-04-22T07:04:10.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7f309cd4a640 1 -- 172.21.15.92:0/2243147083 >> 172.21.15.92:0/2243147083 conn(0x7f309806bfd0 msgr2=0x7f309806f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7fc32f396640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc3200dc700 0x7fc3200dcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7ff43109a640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff4240dd3c0 0x7ff424046460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7fc32f396640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3200dd070 con 0x7fc3200d5200 2024-04-22T07:04:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7fc32f396640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fc3200080e0 con 0x7fc3200dc700 2024-04-22T07:04:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7fc32d933640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc3200dc700 0x7fc3200dcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7fc32d933640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc3200dc700 0x7fc3200dcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36054/0 (socket says 172.21.15.92:36054) 2024-04-22T07:04:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7f309cd4a640 1 -- 172.21.15.92:0/2243147083 shutdown_connections 2024-04-22T07:04:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7f309cd4a640 1 -- 172.21.15.92:0/2243147083 wait complete. 2024-04-22T07:04:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7f222f1ca640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f22200e0840 0x7f222004e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7f222f1ca640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f222000abf0 con 0x7f22200476c0 2024-04-22T07:04:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7f222f1ca640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f222004d060 con 0x7f22200d7090 2024-04-22T07:04:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7f222f1ca640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f222004d820 con 0x7f22200e0840 2024-04-22T07:04:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7f222e1c8640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f22200d7090 0x7f222004edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:55134/0 (socket says 172.21.15.92:55134) 2024-04-22T07:04:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7f222e1c8640 1 -- 172.21.15.92:0/806695558 learned_addr learned my addr 172.21.15.92:0/806695558 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7fc32d933640 1 -- 172.21.15.92:0/1552727465 learned_addr learned my addr 172.21.15.92:0/1552727465 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7fc32c931640 1 --2- 172.21.15.92:0/1552727465 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3200db210 0x7fc320006ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7fc32f396640 1 -- 172.21.15.92:0/1552727465 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc3200071f0 con 0x7fc3200db210 2024-04-22T07:04:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7ff43109a640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4240e37a0 0x7ff42404be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7ff43109a640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4240dd7a0 con 0x7ff42404ba60 2024-04-22T07:04:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7ff43109a640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff424045450 con 0x7ff4240dd3c0 2024-04-22T07:04:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7ff43109a640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff424045fd0 con 0x7ff4240e37a0 2024-04-22T07:04:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7fb0e4bb1640 1 Processor -- start 2024-04-22T07:04:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7fb0e4bb1640 1 -- start start 2024-04-22T07:04:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.246+0000 7f2da6c06640 1 -- start start 2024-04-22T07:04:10.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7fb0e4bb1640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d8050770 0x7fb0d80e3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7f2da6c06640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2da006eb00 0x7f2da015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7f2da6c06640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2da00c7630 0x7f2da00c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7ff42b59e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4240e37a0 0x7ff42404be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7ff42b59e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4240e37a0 0x7ff42404be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52256/0 (socket says 172.21.15.92:52256) 2024-04-22T07:04:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7ff42b59e640 1 -- 172.21.15.92:0/1031965291 learned_addr learned my addr 172.21.15.92:0/1031965291 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7ff42ad9d640 1 --2- 172.21.15.92:0/1031965291 >> v2:172.21.15.145:3300/0 conn(0x7ff4240dd3c0 0x7ff424046460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7f2da6c06640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2da00c7fe0 con 0x7f2da0159250 2024-04-22T07:04:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7f2da6c06640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f2da015fff0 con 0x7f2da00c7630 2024-04-22T07:04:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7f2da6c06640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2da00c6ec0 con 0x7f2da006eb00 2024-04-22T07:04:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7fb0de59c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d8050770 0x7fb0d80e3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7fb0de59c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d8050770 0x7fb0d80e3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52268/0 (socket says 172.21.15.92:52268) 2024-04-22T07:04:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7fb0e4bb1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0d80dd860 0x7fb0d804cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7ff42b59e640 1 -- 172.21.15.92:0/1031965291 >> v2:172.21.15.145:3300/0 conn(0x7ff4240dd3c0 msgr2=0x7ff424046460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7fb0ddd9b640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fb0d80dd400 0x7fb0d80071b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:55154/0 (socket says 172.21.15.92:55154) 2024-04-22T07:04:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fb0ded9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0d80dd860 0x7fb0d804cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fb0de59c640 1 -- 172.21.15.92:0/886636606 learned_addr learned my addr 172.21.15.92:0/886636606 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7fc31ffff640 1 -- 172.21.15.92:0/1552727465 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3174545453 0 0) 0x7fc3200dd070 con 0x7fc3200d5200 2024-04-22T07:04:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fc31ffff640 1 -- 172.21.15.92:0/1552727465 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc314003520 con 0x7fc3200d5200 2024-04-22T07:04:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fc32d933640 1 -- 172.21.15.92:0/1552727465 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3200db210 msgr2=0x7fc320006ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fc32d933640 1 --2- 172.21.15.92:0/1552727465 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3200db210 0x7fc320006ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fc32d933640 1 -- 172.21.15.92:0/1552727465 >> v1:172.21.15.92:6789/0 conn(0x7fc3200d5200 legacy=0x7fc32004b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fc32d933640 1 -- 172.21.15.92:0/1552727465 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3200074d0 con 0x7fc3200dc700 2024-04-22T07:04:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/886636606 --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0d80e42f0 con 0x7fb0d80dd400 2024-04-22T07:04:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7ff42b59e640 1 --2- 172.21.15.92:0/1031965291 >> v2:172.21.15.145:3300/0 conn(0x7ff4240dd3c0 0x7ff424046460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7ff42b59e640 1 -- 172.21.15.92:0/1031965291 >> v1:172.21.15.145:6789/0 conn(0x7ff42404ba60 legacy=0x7ff4240e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7ff42b59e640 1 -- 172.21.15.92:0/1031965291 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4240e3c10 con 0x7ff4240e37a0 2024-04-22T07:04:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7ff42ad9d640 1 --2- 172.21.15.92:0/1031965291 >> v2:172.21.15.145:3300/0 conn(0x7ff4240dd3c0 0x7ff424046460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/886636606 --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fb0d804f1b0 con 0x7fb0d80dd860 2024-04-22T07:04:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1219490139 0 0) 0x7f222004d060 con 0x7f22200d7090 2024-04-22T07:04:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2214003520 con 0x7f22200d7090 2024-04-22T07:04:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2318202606 0 0) 0x7f222000abf0 con 0x7f22200476c0 2024-04-22T07:04:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f222004d060 con 0x7f22200476c0 2024-04-22T07:04:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.247+0000 7f222e9c9640 1 --2- 172.21.15.92:0/806695558 >> v2:172.21.15.92:3300/0 conn(0x7f22200e0840 0x7f222004e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 601118106 0 0) 0x7f222004d060 con 0x7f22200476c0 2024-04-22T07:04:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 >> v2:172.21.15.92:3300/0 conn(0x7f22200e0840 msgr2=0x7f222004e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/886636606 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d804f490 con 0x7fb0d8050770 2024-04-22T07:04:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fb0ded9d640 1 -- 172.21.15.92:0/886636606 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d8050770 msgr2=0x7fb0d80e3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 --2- 172.21.15.92:0/806695558 >> v2:172.21.15.92:3300/0 conn(0x7f22200e0840 0x7f222004e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 >> v1:172.21.15.145:6789/0 conn(0x7f22200d7090 legacy=0x7f222004edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222004db00 con 0x7f22200476c0 2024-04-22T07:04:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f22180015a0 con 0x7f22200476c0 2024-04-22T07:04:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 >> v1:172.21.15.92:6789/0 conn(0x7f22200476c0 legacy=0x7f22200e0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 --2- 172.21.15.92:0/806695558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2214004bf0 0x7f2214004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 --2- 172.21.15.92:0/806695558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22140037a0 0x7f2214003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 --2- 172.21.15.92:0/806695558 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2214004280 0x7f221400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f222004d820 con 0x7f22140037a0 2024-04-22T07:04:10.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f221400c230 con 0x7f2214004bf0 2024-04-22T07:04:10.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f221400c430 con 0x7f2214004280 2024-04-22T07:04:10.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f309cd4a640 1 Processor -- start 2024-04-22T07:04:10.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f309cd4a640 1 -- start start 2024-04-22T07:04:10.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f2da51a3640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2da00c7630 0x7f2da00c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fb0ded9d640 1 --2- 172.21.15.92:0/886636606 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d8050770 0x7fb0d80e3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fb0ded9d640 1 -- 172.21.15.92:0/886636606 >> v1:172.21.15.145:6789/0 conn(0x7fb0d80dd400 legacy=0x7fb0d80071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7fb0ded9d640 1 -- 172.21.15.92:0/886636606 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d804f770 con 0x7fb0d80dd860 2024-04-22T07:04:10.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f222e9c9640 1 --2- 172.21.15.92:0/806695558 >> v2:172.21.15.92:3300/0 conn(0x7f22200e0840 0x7f222004e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:10.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f2da49a2640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f2da0159250 0x7f2da00cb710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:55162/0 (socket says 172.21.15.92:55162) 2024-04-22T07:04:10.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f2da49a2640 1 -- 172.21.15.92:0/540085743 learned_addr learned my addr 172.21.15.92:0/540085743 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:10.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.248+0000 7f2d9ffff640 1 --2- 172.21.15.92:0/540085743 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2da006eb00 0x7f2da015fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7fb0de59c640 1 --2- 172.21.15.92:0/886636606 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d8050770 0x7fb0d80e3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:10.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f222e1c8640 1 --2- 172.21.15.92:0/806695558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2214004bf0 0x7f2214004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f222e9c9640 1 --2- 172.21.15.92:0/806695558 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2214004280 0x7f221400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7fc32d933640 1 --2- 172.21.15.92:0/1552727465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3200dc700 0x7fc3200dcb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc32806a450 tx=0x7fc3280a8f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f32d6b5c8b8afb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f222e1c8640 1 -- 172.21.15.92:0/806695558 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2214004280 msgr2=0x7f221400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f222e1c8640 1 --2- 172.21.15.92:0/806695558 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2214004280 0x7f221400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f222e1c8640 1 -- 172.21.15.92:0/806695558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22140037a0 msgr2=0x7f2214003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f222e1c8640 1 --2- 172.21.15.92:0/806695558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22140037a0 0x7f2214003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f309cd4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3080008c10 0x7f3098201570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f309cd4a640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f30980b1360 0x7f30980afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f309cd4a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30980c4cd0 0x7f30980c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f309cd4a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30980c1c40 con 0x7f3080008c10 2024-04-22T07:04:10.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f309cd4a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f30980aef40 con 0x7f30980c4cd0 2024-04-22T07:04:10.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f309cd4a640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f30980c1dc0 con 0x7f30980b1360 2024-04-22T07:04:10.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f222e1c8640 1 -- 172.21.15.92:0/806695558 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222004db00 con 0x7f2214004bf0 2024-04-22T07:04:10.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7fc31ffff640 1 -- 172.21.15.92:0/1552727465 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3280b6070 con 0x7fc3200dc700 2024-04-22T07:04:10.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f2d9f7fe640 1 -- 172.21.15.92:0/540085743 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3104287360 0 0) 0x7f2da00c7fe0 con 0x7f2da0159250 2024-04-22T07:04:10.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f2d9f7fe640 1 -- 172.21.15.92:0/540085743 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d88003520 con 0x7f2da0159250 2024-04-22T07:04:10.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f2d9ffff640 1 -- 172.21.15.92:0/540085743 >> v2:172.21.15.92:3300/0 conn(0x7f2da00c7630 msgr2=0x7f2da00c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7fc31ffff640 1 -- 172.21.15.92:0/1552727465 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3280a9df0 con 0x7fc3200dc700 2024-04-22T07:04:10.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f2d9ffff640 1 --2- 172.21.15.92:0/540085743 >> v2:172.21.15.92:3300/0 conn(0x7f2da00c7630 0x7f2da00c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7fc31ffff640 1 -- 172.21.15.92:0/1552727465 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328071040 con 0x7fc3200dc700 2024-04-22T07:04:10.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f3096d9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30980c4cd0 0x7f30980c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f3096d9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30980c4cd0 0x7f30980c5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:53686/0 (socket says 172.21.15.92:53686) 2024-04-22T07:04:10.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f3096d9d640 1 -- 172.21.15.92:0/2392829313 learned_addr learned my addr 172.21.15.92:0/2392829313 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7ff42b59e640 1 --2- 172.21.15.92:0/1031965291 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4240e37a0 0x7ff42404be80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009870 tx=0x7ff41c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47c1b922dff7b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f2d9ffff640 1 -- 172.21.15.92:0/540085743 >> v1:172.21.15.145:6789/0 conn(0x7f2da0159250 legacy=0x7f2da00cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.249+0000 7f2d9ffff640 1 -- 172.21.15.92:0/540085743 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da006f0b0 con 0x7f2da006eb00 2024-04-22T07:04:10.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f222e9c9640 1 -- 172.21.15.92:0/806695558 reap_dead start 2024-04-22T07:04:10.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f3096d9d640 1 -- 172.21.15.92:0/2392829313 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f30980b1360 msgr2=0x7f30980afd40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T07:04:10.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f3096d9d640 1 --2- 172.21.15.92:0/2392829313 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f30980b1360 0x7f30980afd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f3096d9d640 1 -- 172.21.15.92:0/2392829313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3080008c10 msgr2=0x7f3098201570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f3096d9d640 1 --2- 172.21.15.92:0/2392829313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3080008c10 0x7f3098201570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f3096d9d640 1 -- 172.21.15.92:0/2392829313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30980c6330 con 0x7f30980c4cd0 2024-04-22T07:04:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7ff429d9b640 1 -- 172.21.15.92:0/1031965291 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c035020 con 0x7ff4240e37a0 2024-04-22T07:04:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7ff429d9b640 1 -- 172.21.15.92:0/1031965291 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41c031640 con 0x7ff4240e37a0 2024-04-22T07:04:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f3096d9d640 1 --2- 172.21.15.92:0/2392829313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30980c4cd0 0x7f30980c5150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f309806ce50 tx=0x7f3090011580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7fb0ded9d640 1 --2- 172.21.15.92:0/886636606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d80dd860 0x7fb0d804cac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0000b60 tx=0x7fb0e00a7c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f89f7301784014 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7ff429d9b640 1 -- 172.21.15.92:0/1031965291 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c02d040 con 0x7ff4240e37a0 2024-04-22T07:04:10.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f2d9ffff640 1 --2- 172.21.15.92:0/540085743 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2da006eb00 0x7f2da015fab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009d90 tx=0x7f2d9802bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3403f2b90e839b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f222e1c8640 1 --2- 172.21.15.92:0/806695558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2214004bf0 0x7f2214004ff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2224007fd0 tx=0x7f2214005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9465b48a70cb734c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.250+0000 7f3086ffd640 1 -- 172.21.15.92:0/2392829313 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309003f070 con 0x7f30980c4cd0 2024-04-22T07:04:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7f309cd4a640 1 -- 172.21.15.92:0/2392829313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30980ca220 con 0x7f30980c4cd0 2024-04-22T07:04:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7f309cd4a640 1 -- 172.21.15.92:0/2392829313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30980ca7e0 con 0x7f30980c4cd0 2024-04-22T07:04:10.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7f2d9f7fe640 1 -- 172.21.15.92:0/540085743 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98039070 con 0x7f2da006eb00 2024-04-22T07:04:10.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7f2d9f7fe640 1 -- 172.21.15.92:0/540085743 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9802cce0 con 0x7f2da006eb00 2024-04-22T07:04:10.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7fc32f396640 1 -- 172.21.15.92:0/1552727465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3200dc700 msgr2=0x7fc3200dcb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7fc32f396640 1 --2- 172.21.15.92:0/1552727465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3200dc700 0x7fc3200dcb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc32806a450 tx=0x7fc3280a8f40 comp rx=0 tx=0).stop 2024-04-22T07:04:10.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7fc32f396640 1 -- 172.21.15.92:0/1552727465 shutdown_connections 2024-04-22T07:04:10.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7fc32f396640 1 --2- 172.21.15.92:0/1552727465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3200dc700 0x7fc3200dcb30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7fc32f396640 1 --2- 172.21.15.92:0/1552727465 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3200db210 0x7fc320006ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7fc32f396640 1 -- 172.21.15.92:0/1552727465 >> 172.21.15.92:0/1552727465 conn(0x7fc32004d4b0 msgr2=0x7fc32004d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7ff43109a640 1 -- 172.21.15.92:0/1031965291 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4240e37a0 msgr2=0x7ff42404be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7ff43109a640 1 --2- 172.21.15.92:0/1031965291 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4240e37a0 0x7ff42404be80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009870 tx=0x7ff41c02b8f0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7ff43109a640 1 -- 172.21.15.92:0/1031965291 shutdown_connections 2024-04-22T07:04:10.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7ff43109a640 1 --2- 172.21.15.92:0/1031965291 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4240e37a0 0x7ff42404be80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7ff43109a640 1 --2- 172.21.15.92:0/1031965291 >> v2:172.21.15.145:3300/0 conn(0x7ff4240dd3c0 0x7ff424046460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7ff43109a640 1 -- 172.21.15.92:0/1031965291 >> 172.21.15.92:0/1031965291 conn(0x7ff424050180 msgr2=0x7ff424050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7ff43109a640 1 -- 172.21.15.92:0/1031965291 shutdown_connections 2024-04-22T07:04:10.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7fb0dd59a640 1 -- 172.21.15.92:0/886636606 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00b5070 con 0x7fb0d80dd860 2024-04-22T07:04:10.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7fb0dd59a640 1 -- 172.21.15.92:0/886636606 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e00a8880 con 0x7fb0d80dd860 2024-04-22T07:04:10.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7fb0dd59a640 1 -- 172.21.15.92:0/886636606 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00a8cf0 con 0x7fb0d80dd860 2024-04-22T07:04:10.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2224037070 con 0x7f2214004bf0 2024-04-22T07:04:10.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7f222d1c6640 1 -- 172.21.15.92:0/806695558 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2224004aa0 con 0x7f2214004bf0 2024-04-22T07:04:10.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f222f1ca640 1 -- 172.21.15.92:0/806695558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2214004bf0 msgr2=0x7f2214004ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f222f1ca640 1 --2- 172.21.15.92:0/806695558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2214004bf0 0x7f2214004ff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2224007fd0 tx=0x7f2214005830 comp rx=0 tx=0).stop 2024-04-22T07:04:10.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/886636606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d80dd860 msgr2=0x7fb0d804cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7fb0e4bb1640 1 --2- 172.21.15.92:0/886636606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d80dd860 0x7fb0d804cac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0000b60 tx=0x7fb0e00a7c20 comp rx=0 tx=0).stop 2024-04-22T07:04:10.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7f3086ffd640 1 -- 172.21.15.92:0/2392829313 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3090011e10 con 0x7f30980c4cd0 2024-04-22T07:04:10.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f3086ffd640 1 -- 172.21.15.92:0/2392829313 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309000b230 con 0x7f30980c4cd0 2024-04-22T07:04:10.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7f2d9f7fe640 1 -- 172.21.15.92:0/540085743 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98039070 con 0x7f2da006eb00 2024-04-22T07:04:10.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.251+0000 7ff43109a640 1 -- 172.21.15.92:0/1031965291 wait complete. 2024-04-22T07:04:10.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f222f1ca640 1 -- 172.21.15.92:0/806695558 shutdown_connections 2024-04-22T07:04:10.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f222f1ca640 1 --2- 172.21.15.92:0/806695558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2214004bf0 0x7f2214004ff0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f222f1ca640 1 -- 172.21.15.92:0/806695558 >> 172.21.15.92:0/806695558 conn(0x7f222000e6a0 msgr2=0x7f2220009d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/886636606 shutdown_connections 2024-04-22T07:04:10.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7fb0e4bb1640 1 --2- 172.21.15.92:0/886636606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d80dd860 0x7fb0d804cac0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7fb0e4bb1640 1 --2- 172.21.15.92:0/886636606 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d8050770 0x7fb0d80e3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/886636606 >> 172.21.15.92:0/886636606 conn(0x7fb0d800e160 msgr2=0x7fb0d800e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7fc32f396640 1 -- 172.21.15.92:0/1552727465 shutdown_connections 2024-04-22T07:04:10.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/886636606 shutdown_connections 2024-04-22T07:04:10.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f222f1ca640 1 -- 172.21.15.92:0/806695558 shutdown_connections 2024-04-22T07:04:10.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f222f1ca640 1 -- 172.21.15.92:0/806695558 wait complete. 2024-04-22T07:04:10.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.253+0000 7f3086ffd640 1 -- 172.21.15.92:0/2392829313 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3090042070 con 0x7f30980c4cd0 2024-04-22T07:04:10.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/886636606 wait complete. 2024-04-22T07:04:10.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f2da6c06640 1 -- 172.21.15.92:0/540085743 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2da006eb00 msgr2=0x7f2da015fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f2da6c06640 1 --2- 172.21.15.92:0/540085743 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2da006eb00 0x7f2da015fab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009d90 tx=0x7f2d9802bf30 comp rx=0 tx=0).stop 2024-04-22T07:04:10.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f2da6c06640 1 -- 172.21.15.92:0/540085743 shutdown_connections 2024-04-22T07:04:10.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f2da6c06640 1 --2- 172.21.15.92:0/540085743 >> v2:172.21.15.92:3300/0 conn(0x7f2da00c7630 0x7f2da00c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f2da6c06640 1 --2- 172.21.15.92:0/540085743 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2da006eb00 0x7f2da015fab0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009d90 tx=0x7f2d9802bf30 comp rx=0 tx=0).stop 2024-04-22T07:04:10.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f2da6c06640 1 -- 172.21.15.92:0/540085743 >> 172.21.15.92:0/540085743 conn(0x7f2da00cc220 msgr2=0x7f2da00cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f2da6c06640 1 -- 172.21.15.92:0/540085743 shutdown_connections 2024-04-22T07:04:10.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.252+0000 7f2da6c06640 1 -- 172.21.15.92:0/540085743 wait complete. 2024-04-22T07:04:10.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.254+0000 7ff43109a640 1 Processor -- start 2024-04-22T07:04:10.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.254+0000 7ff43109a640 1 -- start start 2024-04-22T07:04:10.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.254+0000 7f222f1ca640 1 Processor -- start 2024-04-22T07:04:10.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.254+0000 7f222f1ca640 1 -- start start 2024-04-22T07:04:10.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.254+0000 7f3086ffd640 1 --2- 172.21.15.92:0/2392829313 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f307806c9f0 0x7f307806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.254+0000 7f309cd4a640 1 -- 172.21.15.92:0/2392829313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f307c001030 con 0x7f30980c4cd0 2024-04-22T07:04:10.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.254+0000 7f2da6c06640 1 Processor -- start 2024-04-22T07:04:10.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.254+0000 7fc32f396640 1 -- 172.21.15.92:0/1552727465 wait complete. 2024-04-22T07:04:10.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.254+0000 7f309659c640 1 --2- 172.21.15.92:0/2392829313 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f307806c9f0 0x7f307806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7fb0e4bb1640 1 Processor -- start 2024-04-22T07:04:10.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f3086ffd640 1 -- 172.21.15.92:0/2392829313 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f30900ad170 con 0x7f30980c4cd0 2024-04-22T07:04:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f3086ffd640 1 --2- 172.21.15.92:0/2392829313 >> [v2:172.21.15.92:6800/24674040,v1:172.21.15.92:6801/24674040] conn(0x7f3078074bf0 0x7f3078077010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f3086ffd640 1 -- 172.21.15.92:0/2392829313 --> [v2:172.21.15.92:6800/24674040,v1:172.21.15.92:6801/24674040] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3090009060 con 0x7f3078074bf0 2024-04-22T07:04:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.254+0000 7f2da6c06640 1 -- start start 2024-04-22T07:04:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f2da6c06640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00c7630 0x7f2da015d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f2da6c06640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2da0159250 0x7f2da0200f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f2da6c06640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2da01ef440 0x7f2da02014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2da6c06640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2da006f950 con 0x7f2da00c7630 2024-04-22T07:04:10.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2da6c06640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2da006f670 con 0x7f2da0159250 2024-04-22T07:04:10.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7fb0e4bb1640 1 -- start start 2024-04-22T07:04:10.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7fb0e4bb1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d8050770 0x7fb0d81827e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f3095d9b640 1 --2- 172.21.15.92:0/2392829313 >> [v2:172.21.15.92:6800/24674040,v1:172.21.15.92:6801/24674040] conn(0x7f3078074bf0 0x7f3078077010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2da6c06640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2da00c7110 con 0x7f2da01ef440 2024-04-22T07:04:10.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7fb0e4bb1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d80dd400 0x7fb0d80e6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2da49a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00c7630 0x7f2da015d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2da49a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00c7630 0x7f2da015d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36108/0 (socket says 172.21.15.92:36108) 2024-04-22T07:04:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2da49a2640 1 -- 172.21.15.92:0/2832159377 learned_addr learned my addr 172.21.15.92:0/2832159377 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2da51a3640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2da01ef440 0x7f2da02014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2da49a2640 1 -- 172.21.15.92:0/2832159377 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2da01ef440 msgr2=0x7f2da02014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7fb0de59c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d8050770 0x7fb0d81827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7fb0de59c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d8050770 0x7fb0d81827e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:53708/0 (socket says 172.21.15.92:53708) 2024-04-22T07:04:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7fb0de59c640 1 -- 172.21.15.92:0/1745708359 learned_addr learned my addr 172.21.15.92:0/1745708359 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7fb0e4bb1640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d80dd860 0x7fb0d80e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2da49a2640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2da01ef440 0x7f2da02014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8050010 con 0x7fb0d80dd400 2024-04-22T07:04:10.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d804fd30 con 0x7fb0d8050770 2024-04-22T07:04:10.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d804ee90 con 0x7fb0d80dd860 2024-04-22T07:04:10.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7fb0ddd9b640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d80dd400 0x7fb0d80e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2d9ffff640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2da0159250 0x7f2da0200f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2da49a2640 1 -- 172.21.15.92:0/2832159377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2da0159250 msgr2=0x7f2da0200f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7fb0ded9d640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d80dd860 0x7fb0d80e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2da49a2640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2da0159250 0x7f2da0200f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f2da49a2640 1 -- 172.21.15.92:0/2832159377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da01f3160 con 0x7f2da00c7630 2024-04-22T07:04:10.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7f2da49a2640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00c7630 0x7f2da015d400 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400c410 tx=0x7f2d9400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7fc32f396640 1 Processor -- start 2024-04-22T07:04:10.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f885f6ba640 1 Processor -- start 2024-04-22T07:04:10.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f885f6ba640 1 -- start start 2024-04-22T07:04:10.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7fc32f396640 1 -- start start 2024-04-22T07:04:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7fb0ddd9b640 1 -- 172.21.15.92:0/1745708359 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d80dd860 msgr2=0x7fb0d80e49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7fb0ddd9b640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d80dd860 0x7fb0d80e49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7fb0ddd9b640 1 -- 172.21.15.92:0/1745708359 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d8050770 msgr2=0x7fb0d81827e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7fb0ddd9b640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d8050770 0x7fb0d81827e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7fb0ddd9b640 1 -- 172.21.15.92:0/1745708359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d8182d20 con 0x7fb0d80dd400 2024-04-22T07:04:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7fb0ddd9b640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d80dd400 0x7fb0d80e6810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400c450 tx=0x7fb0d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7f2d9d7fa640 1 -- 172.21.15.92:0/2832159377 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94017070 con 0x7f2da00c7630 2024-04-22T07:04:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f222f1ca640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2214004bf0 0x7f22200d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7f2da6c06640 1 -- 172.21.15.92:0/2832159377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da01f3440 con 0x7f2da00c7630 2024-04-22T07:04:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f222f1ca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2220173fa0 0x7f22200cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f222f1ca640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22200cdd90 0x7f22200ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f222f1ca640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f222004e6a0 con 0x7f2220173fa0 2024-04-22T07:04:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7f2da6c06640 1 -- 172.21.15.92:0/2832159377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da00c4c00 con 0x7f2da00c7630 2024-04-22T07:04:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f222f1ca640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f222004cd20 con 0x7f22200cdd90 2024-04-22T07:04:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f222f1ca640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f222004e820 con 0x7f2214004bf0 2024-04-22T07:04:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f222d9c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2220173fa0 0x7f22200cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f222d9c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2220173fa0 0x7f22200cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36106/0 (socket says 172.21.15.92:36106) 2024-04-22T07:04:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f222d9c7640 1 -- 172.21.15.92:0/1512330089 learned_addr learned my addr 172.21.15.92:0/1512330089 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.255+0000 7f222e1c8640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2214004bf0 0x7f22200d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f222e9c9640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22200cdd90 0x7f22200ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f222d9c7640 1 -- 172.21.15.92:0/1512330089 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2214004bf0 msgr2=0x7f22200d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f222d9c7640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2214004bf0 0x7f22200d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f222d9c7640 1 -- 172.21.15.92:0/1512330089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22200cdd90 msgr2=0x7f22200ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f222d9c7640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22200cdd90 0x7f22200ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f222d9c7640 1 -- 172.21.15.92:0/1512330089 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22200d1490 con 0x7f2220173fa0 2024-04-22T07:04:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f222e9c9640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22200cdd90 0x7f22200ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7f222d9c7640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2220173fa0 0x7f22200cf030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f221800ca40 tx=0x7f221800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.256+0000 7f3095d9b640 1 --2- 172.21.15.92:0/2392829313 >> [v2:172.21.15.92:6800/24674040,v1:172.21.15.92:6801/24674040] conn(0x7f3078074bf0 0x7f3078077010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7f309659c640 1 --2- 172.21.15.92:0/2392829313 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f307806c9f0 0x7f307806eeb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f30980c9ba0 tx=0x7f308c008230 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7f885f6ba640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88580c1c40 0x7f88580ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7f885f6ba640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f88580aee30 0x7f885806dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7f885f6ba640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88580c6ef0 con 0x7f885806e260 2024-04-22T07:04:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7f885f6ba640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f885806e710 con 0x7f88580aee30 2024-04-22T07:04:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7f885f6ba640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f88580c71d0 con 0x7f88580c1c40 2024-04-22T07:04:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7f885d456640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88580c1c40 0x7f88580ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7f885d456640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88580c1c40 0x7f88580ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52320/0 (socket says 172.21.15.92:52320) 2024-04-22T07:04:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7f885d456640 1 -- 172.21.15.92:0/1611042391 learned_addr learned my addr 172.21.15.92:0/1611042391 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7f885cc55640 1 --2- 172.21.15.92:0/1611042391 >> v2:172.21.15.145:3300/0 conn(0x7f88580aee30 0x7f885806dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7f3086ffd640 1 -- 172.21.15.92:0/2392829313 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3090076aa0 con 0x7f30980c4cd0 2024-04-22T07:04:10.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7f3086ffd640 1 -- 172.21.15.92:0/2392829313 <== osd.3 v2:172.21.15.92:6800/24674040 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3090009060 con 0x7f3078074bf0 2024-04-22T07:04:10.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7ff43109a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff42404ba60 0x7ff424179a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7ff43109a640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4240dd3c0 0x7ff4240e0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7ff43109a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4240e37a0 0x7ff4240d8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fb0ceffd640 1 -- 172.21.15.92:0/1745708359 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4017070 con 0x7fb0d80dd400 2024-04-22T07:04:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7ff43109a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff424045980 con 0x7ff42404ba60 2024-04-22T07:04:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7ff43109a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff4240456a0 con 0x7ff4240e37a0 2024-04-22T07:04:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7ff43109a640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff424045c60 con 0x7ff4240dd3c0 2024-04-22T07:04:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fc32f396640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3200d5200 0x7fc32004f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fc32f396640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3200db210 0x7fc320178b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fc32f396640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3200dc700 0x7fc3200cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fc32d132640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3200d5200 0x7fc32004f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fc32d132640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3200d5200 0x7fc32004f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36150/0 (socket says 172.21.15.92:36150) 2024-04-22T07:04:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fc32d132640 1 -- 172.21.15.92:0/2492677980 learned_addr learned my addr 172.21.15.92:0/2492677980 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc320007d70 con 0x7fc3200d5200 2024-04-22T07:04:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc320007a90 con 0x7fc3200db210 2024-04-22T07:04:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc3200d55e0 con 0x7fc3200dc700 2024-04-22T07:04:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7ff42ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff42404ba60 0x7ff424179a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fc32c931640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3200db210 0x7fc320178b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fc32d933640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3200dc700 0x7fc3200cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7f2d9d7fa640 1 -- 172.21.15.92:0/2832159377 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d94005ce0 con 0x7f2da00c7630 2024-04-22T07:04:10.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7ff42ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff42404ba60 0x7ff424179a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36134/0 (socket says 172.21.15.92:36134) 2024-04-22T07:04:10.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.257+0000 7ff42ad9d640 1 -- 172.21.15.92:0/3482513486 learned_addr learned my addr 172.21.15.92:0/3482513486 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7ff42ad9d640 1 -- 172.21.15.92:0/3482513486 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4240dd3c0 msgr2=0x7ff4240e0510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:10.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7efcd091c640 1 Processor -- start 2024-04-22T07:04:10.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7ff42ad9d640 1 --2- 172.21.15.92:0/3482513486 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4240dd3c0 0x7ff4240e0510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7ff42ad9d640 1 -- 172.21.15.92:0/3482513486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4240e37a0 msgr2=0x7ff4240d8020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:10.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7ff42ad9d640 1 --2- 172.21.15.92:0/3482513486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4240e37a0 0x7ff4240d8020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7ff42ad9d640 1 -- 172.21.15.92:0/3482513486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42417c770 con 0x7ff42404ba60 2024-04-22T07:04:10.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7ff42ad9d640 1 --2- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff42404ba60 0x7ff424179a70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff42000c7f0 tx=0x7ff42000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.259+0000 7efcd091c640 1 -- start start 2024-04-22T07:04:10.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7ff4137fe640 1 -- 172.21.15.92:0/3482513486 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420015070 con 0x7ff42404ba60 2024-04-22T07:04:10.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7ff43109a640 1 -- 172.21.15.92:0/3482513486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42417ca10 con 0x7ff42404ba60 2024-04-22T07:04:10.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7ff43109a640 1 -- 172.21.15.92:0/3482513486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42417cfb0 con 0x7ff42404ba60 2024-04-22T07:04:10.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7f885d456640 1 -- 172.21.15.92:0/1611042391 >> v2:172.21.15.145:3300/0 conn(0x7f88580aee30 msgr2=0x7f885806dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d80e2e50 con 0x7fb0d80dd400 2024-04-22T07:04:10.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d80e30f0 con 0x7fb0d80dd400 2024-04-22T07:04:10.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.259+0000 7fb0ceffd640 1 -- 172.21.15.92:0/1745708359 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d40074e0 con 0x7fb0d80dd400 2024-04-22T07:04:10.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.259+0000 7fb0ceffd640 1 -- 172.21.15.92:0/1745708359 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4007690 con 0x7fb0d80dd400 2024-04-22T07:04:10.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7f885d456640 1 --2- 172.21.15.92:0/1611042391 >> v2:172.21.15.145:3300/0 conn(0x7f88580aee30 0x7f885806dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7f885d456640 1 -- 172.21.15.92:0/1611042391 >> v1:172.21.15.92:6789/0 conn(0x7f885806e260 legacy=0x7f885816ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7f885d456640 1 -- 172.21.15.92:0/1611042391 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885806d520 con 0x7f88580c1c40 2024-04-22T07:04:10.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.259+0000 7f885d456640 1 --2- 172.21.15.92:0/1611042391 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88580c1c40 0x7f88580ae8f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8854009870 tx=0x7f885402fe40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38359abfadafb875 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.259+0000 7f2d9d7fa640 1 -- 172.21.15.92:0/2832159377 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d940072f0 con 0x7f2da00c7630 2024-04-22T07:04:10.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.259+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb0c4001030 con 0x7fb0d80dd400 2024-04-22T07:04:10.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.259+0000 7ff43109a640 1 -- 172.21.15.92:0/3482513486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff424005390 con 0x7ff42404ba60 2024-04-22T07:04:10.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.258+0000 7f221effd640 1 -- 172.21.15.92:0/1512330089 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218005270 con 0x7f2220173fa0 2024-04-22T07:04:10.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.259+0000 7f221effd640 1 -- 172.21.15.92:0/1512330089 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2218005430 con 0x7f2220173fa0 2024-04-22T07:04:10.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.259+0000 7fc32d933640 1 -- 172.21.15.92:0/2492677980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3200db210 msgr2=0x7fc320178b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7fc32d933640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3200db210 0x7fc320178b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7fc32d933640 1 -- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3200d5200 msgr2=0x7fc32004f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7fc32d933640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3200d5200 0x7fc32004f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7fc32d933640 1 -- 172.21.15.92:0/2492677980 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3200d7de0 con 0x7fc3200dc700 2024-04-22T07:04:10.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7fc32c931640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3200db210 0x7fc320178b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:10.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7ff4137fe640 1 -- 172.21.15.92:0/3482513486 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff420007a30 con 0x7ff42404ba60 2024-04-22T07:04:10.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7fc32d132640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3200d5200 0x7fc32004f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:10.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7f221effd640 1 -- 172.21.15.92:0/1512330089 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221800a940 con 0x7f2220173fa0 2024-04-22T07:04:10.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7fc32d933640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3200dc700 0x7fc3200cd5a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc328050a90 tx=0x7fc3280ab070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7f222f1ca640 1 -- 172.21.15.92:0/1512330089 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22200d1770 con 0x7f2220173fa0 2024-04-22T07:04:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7f222f1ca640 1 -- 172.21.15.92:0/1512330089 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22200d1d30 con 0x7f2220173fa0 2024-04-22T07:04:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7fc31dffb640 1 -- 172.21.15.92:0/2492677980 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3280c1070 con 0x7fc3200dc700 2024-04-22T07:04:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7fc31dffb640 1 -- 172.21.15.92:0/2492677980 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3280a9bb0 con 0x7fc3200dc700 2024-04-22T07:04:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7efcd091c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40de3b0 0x7efcc40de840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7efcd091c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc4006e90 con 0x7efcc4045720 2024-04-22T07:04:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7efcd091c640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc4007170 con 0x7efcc40d5090 2024-04-22T07:04:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7efcd091c640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efcc4007aa0 con 0x7efcc40de3b0 2024-04-22T07:04:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7efccbfff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40de3b0 0x7efcc40de840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7f884ffff640 1 -- 172.21.15.92:0/1611042391 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885403d070 con 0x7f88580c1c40 2024-04-22T07:04:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7fc31dffb640 1 -- 172.21.15.92:0/2492677980 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3280c1070 con 0x7fc3200dc700 2024-04-22T07:04:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7f884ffff640 1 -- 172.21.15.92:0/1611042391 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8854030bf0 con 0x7f88580c1c40 2024-04-22T07:04:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.261+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3200d94b0 con 0x7fc3200dc700 2024-04-22T07:04:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7efccbfff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40de3b0 0x7efcc40de840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52336/0 (socket says 172.21.15.92:52336) 2024-04-22T07:04:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.261+0000 7efccbfff640 1 -- 172.21.15.92:0/2656469528 learned_addr learned my addr 172.21.15.92:0/2656469528 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.261+0000 7ff4137fe640 1 -- 172.21.15.92:0/3482513486 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420007de0 con 0x7ff42404ba60 2024-04-22T07:04:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.261+0000 7f884ffff640 1 -- 172.21.15.92:0/1611042391 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854038440 con 0x7f88580c1c40 2024-04-22T07:04:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.260+0000 7f2da6c06640 1 -- 172.21.15.92:0/2832159377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d80001030 con 0x7f2da00c7630 2024-04-22T07:04:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.261+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3200d8050 con 0x7fc3200dc700 2024-04-22T07:04:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.261+0000 7f2d9d7fa640 1 -- 172.21.15.92:0/2832159377 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d9401a070 con 0x7f2da00c7630 2024-04-22T07:04:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.261+0000 7efcca7fc640 1 -- 172.21.15.92:0/2656469528 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787003790 0 0) 0x7efcc4007170 con 0x7efcc40d5090 2024-04-22T07:04:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.261+0000 7efcca7fc640 1 -- 172.21.15.92:0/2656469528 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcbc003520 con 0x7efcc40d5090 2024-04-22T07:04:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.262+0000 7efccbfff640 1 -- 172.21.15.92:0/2656469528 >> v1:172.21.15.145:6789/0 conn(0x7efcc40d5090 legacy=0x7efcc404b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.262+0000 7efccbfff640 1 -- 172.21.15.92:0/2656469528 >> v1:172.21.15.92:6789/0 conn(0x7efcc4045720 legacy=0x7efcc4045b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.262+0000 7efccbfff640 1 -- 172.21.15.92:0/2656469528 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc4007450 con 0x7efcc40de3b0 2024-04-22T07:04:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.262+0000 7f2d9d7fa640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f2d8c06c940 0x7f2d8c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.262+0000 7f2d9d7fa640 1 -- 172.21.15.92:0/2832159377 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2d9408aea0 con 0x7f2da00c7630 2024-04-22T07:04:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.262+0000 7ff4137fe640 1 -- 172.21.15.92:0/3482513486 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff420018070 con 0x7ff42404ba60 2024-04-22T07:04:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.262+0000 7f2d9d7fa640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.145:6804/3070790698,v1:172.21.15.145:6806/3070790698] conn(0x7f2d8c074ae0 0x7f2d8c076f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.262+0000 7f2d9d7fa640 1 -- 172.21.15.92:0/2832159377 --> [v2:172.21.15.145:6804/3070790698,v1:172.21.15.145:6806/3070790698] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d94009e50 con 0x7f2d8c074ae0 2024-04-22T07:04:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.262+0000 7f222f1ca640 1 -- 172.21.15.92:0/1512330089 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f22200d7ee0 con 0x7f2220173fa0 2024-04-22T07:04:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.262+0000 7f2d9ffff640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f2d8c06c940 0x7f2d8c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.261+0000 7f885f6ba640 1 -- 172.21.15.92:0/1611042391 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88580c1c40 msgr2=0x7f88580ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.261+0000 7f885f6ba640 1 --2- 172.21.15.92:0/1611042391 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88580c1c40 0x7f88580ae8f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8854009870 tx=0x7f885402fe40 comp rx=0 tx=0).stop 2024-04-22T07:04:10.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7fc31dffb640 1 -- 172.21.15.92:0/2492677980 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3280ba430 con 0x7fc3200dc700 2024-04-22T07:04:10.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7ff4137fe640 1 --2- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff41806c9f0 0x7ff41806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7ff42a59c640 1 --2- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff41806c9f0 0x7ff41806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7f2d9d7fa640 1 -- 172.21.15.92:0/2832159377 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2d94055050 con 0x7f2da00c7630 2024-04-22T07:04:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7f2da51a3640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.145:6804/3070790698,v1:172.21.15.145:6806/3070790698] conn(0x7f2d8c074ae0 0x7f2d8c076f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7f885f6ba640 1 -- 172.21.15.92:0/1611042391 shutdown_connections 2024-04-22T07:04:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7f885f6ba640 1 --2- 172.21.15.92:0/1611042391 >> v2:172.21.15.145:3300/0 conn(0x7f88580aee30 0x7f885806dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7f885f6ba640 1 --2- 172.21.15.92:0/1611042391 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88580c1c40 0x7f88580ae8f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7f885f6ba640 1 -- 172.21.15.92:0/1611042391 >> 172.21.15.92:0/1611042391 conn(0x7f885806bfd0 msgr2=0x7f885806f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7ff4137fe640 1 -- 172.21.15.92:0/3482513486 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff42008bfc0 con 0x7ff42404ba60 2024-04-22T07:04:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7ff4137fe640 1 --2- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:6824/3408935700,v1:172.21.15.92:6825/3408935700] conn(0x7ff418074c70 0x7ff418077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7ff4137fe640 1 -- 172.21.15.92:0/3482513486 --> [v2:172.21.15.92:6824/3408935700,v1:172.21.15.92:6825/3408935700] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff420007a30 con 0x7ff418074c70 2024-04-22T07:04:10.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7ff4137fe640 1 -- 172.21.15.92:0/3482513486 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff42008c3b0 con 0x7ff42404ba60 2024-04-22T07:04:10.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7fc31dffb640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc30c06ccf0 0x7fc30c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7fc31dffb640 1 -- 172.21.15.92:0/2492677980 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc3280b2e10 con 0x7fc3200dc700 2024-04-22T07:04:10.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7f2d9ffff640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f2d8c06c940 0x7f2d8c06ee00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d98036000 tx=0x7f2d98030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7fb0ceffd640 1 -- 172.21.15.92:0/1745708359 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0d401a070 con 0x7fb0d80dd400 2024-04-22T07:04:10.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7efccbfff640 1 --2- 172.21.15.92:0/2656469528 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40de3b0 0x7efcc40de840 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efcb80098a0 tx=0x7efcb802ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35f5dbf968a2fd45 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7f885f6ba640 1 -- 172.21.15.92:0/1611042391 shutdown_connections 2024-04-22T07:04:10.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7fc32d132640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc30c06ccf0 0x7fc30c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7fc32f396640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:6808/3594806740,v1:172.21.15.92:6809/3594806740] conn(0x7fc308001650 0x7fc308003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 --> [v2:172.21.15.92:6808/3594806740,v1:172.21.15.92:6809/3594806740] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc308006bb0 con 0x7fc308001650 2024-04-22T07:04:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7fb0ceffd640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb0c006c940 0x7fb0c006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7fc32c931640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:6808/3594806740,v1:172.21.15.92:6809/3594806740] conn(0x7fc308001650 0x7fc308003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7fb0de59c640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb0c006c940 0x7fb0c006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7ff42b59e640 1 --2- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:6824/3408935700,v1:172.21.15.92:6825/3408935700] conn(0x7ff418074c70 0x7ff418077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7fb0ceffd640 1 -- 172.21.15.92:0/1745708359 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb0d408b610 con 0x7fb0d80dd400 2024-04-22T07:04:10.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7fb0ceffd640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.145:6800/2981429006,v1:172.21.15.145:6801/2981429006] conn(0x7fb0c0074b60 0x7fb0c0076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7fb0ceffd640 1 -- 172.21.15.92:0/1745708359 --> [v2:172.21.15.145:6800/2981429006,v1:172.21.15.145:6801/2981429006] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0d4014ec0 con 0x7fb0c0074b60 2024-04-22T07:04:10.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7fb0ceffd640 1 -- 172.21.15.92:0/1745708359 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb0d408baf0 con 0x7fb0d80dd400 2024-04-22T07:04:10.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7ff42b59e640 1 --2- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:6824/3408935700,v1:172.21.15.92:6825/3408935700] conn(0x7ff418074c70 0x7ff418077090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7fb0ded9d640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.145:6800/2981429006,v1:172.21.15.145:6801/2981429006] conn(0x7fb0c0074b60 0x7fb0c0076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7fc32c931640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:6808/3594806740,v1:172.21.15.92:6809/3594806740] conn(0x7fc308001650 0x7fc308003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7fb0de59c640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb0c006c940 0x7fb0c006ee00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00099f0 tx=0x7fb0d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.263+0000 7f885f6ba640 1 -- 172.21.15.92:0/1611042391 wait complete. 2024-04-22T07:04:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.264+0000 7ff42a59c640 1 --2- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff41806c9f0 0x7ff41806eeb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff42c050f50 tx=0x7ff42c075de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7efcca7fc640 1 -- 172.21.15.92:0/2656469528 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb802f820 con 0x7efcc40de3b0 2024-04-22T07:04:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7efcca7fc640 1 -- 172.21.15.92:0/2656469528 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb802f9c0 con 0x7efcc40de3b0 2024-04-22T07:04:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7f885f6ba640 1 Processor -- start 2024-04-22T07:04:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7efcca7fc640 1 -- 172.21.15.92:0/2656469528 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb802fd90 con 0x7efcc40de3b0 2024-04-22T07:04:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7f885f6ba640 1 -- start start 2024-04-22T07:04:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7f2da51a3640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.145:6804/3070790698,v1:172.21.15.145:6806/3070790698] conn(0x7f2d8c074ae0 0x7f2d8c076f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7fc32d132640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc30c06ccf0 0x7fc30c06f1b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc324001820 tx=0x7fc324009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7f221effd640 1 -- 172.21.15.92:0/1512330089 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f221800aae0 con 0x7f2220173fa0 2024-04-22T07:04:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7ff4137fe640 1 -- 172.21.15.92:0/3482513486 <== osd.0 v2:172.21.15.92:6824/3408935700 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff420007a30 con 0x7ff418074c70 2024-04-22T07:04:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7fb0ded9d640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.145:6800/2981429006,v1:172.21.15.145:6801/2981429006] conn(0x7fb0c0074b60 0x7fb0c0076f80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7fc31dffb640 1 -- 172.21.15.92:0/2492677980 <== osd.2 v2:172.21.15.92:6808/3594806740 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc308006bb0 con 0x7fc308001650 2024-04-22T07:04:10.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.265+0000 7f2d9d7fa640 1 -- 172.21.15.92:0/2832159377 <== osd.4 v2:172.21.15.145:6804/3070790698 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d94009e50 con 0x7f2d8c074ae0 2024-04-22T07:04:10.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f221effd640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f220c06c940 0x7f220c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f221effd640 1 -- 172.21.15.92:0/1512330089 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f221808c2b0 con 0x7f2220173fa0 2024-04-22T07:04:10.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f221effd640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:6816/2805470585,v1:172.21.15.92:6817/2805470585] conn(0x7f220c074bc0 0x7f220c076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7efcd091c640 1 -- 172.21.15.92:0/2656469528 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40de3b0 msgr2=0x7efcc40de840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7efcd091c640 1 --2- 172.21.15.92:0/2656469528 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40de3b0 0x7efcc40de840 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efcb80098a0 tx=0x7efcb802ba00 comp rx=0 tx=0).stop 2024-04-22T07:04:10.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f885f6ba640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f885806e260 0x7f88581fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f221effd640 1 -- 172.21.15.92:0/1512330089 --> [v2:172.21.15.92:6816/2805470585,v1:172.21.15.92:6817/2805470585] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f220c077690 con 0x7f220c074bc0 2024-04-22T07:04:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f222e1c8640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f220c06c940 0x7f220c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f221effd640 1 -- 172.21.15.92:0/1512330089 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f221808c790 con 0x7f2220173fa0 2024-04-22T07:04:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f222e9c9640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:6816/2805470585,v1:172.21.15.92:6817/2805470585] conn(0x7f220c074bc0 0x7f220c076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f885f6ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88580c1c40 0x7f8858161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7efcd091c640 1 -- 172.21.15.92:0/2656469528 shutdown_connections 2024-04-22T07:04:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7fb0ceffd640 1 -- 172.21.15.92:0/1745708359 <== osd.5 v2:172.21.15.145:6800/2981429006 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0d4014ec0 con 0x7fb0c0074b60 2024-04-22T07:04:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7efcd091c640 1 --2- 172.21.15.92:0/2656469528 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40de3b0 0x7efcc40de840 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.267+0000 7efcd091c640 1 -- 172.21.15.92:0/2656469528 >> 172.21.15.92:0/2656469528 conn(0x7efcc40505d0 msgr2=0x7efcc40509e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.267+0000 7f222e9c9640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:6816/2805470585,v1:172.21.15.92:6817/2805470585] conn(0x7f220c074bc0 0x7f220c076fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f885f6ba640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88581f1590 0x7f88581f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.267+0000 7efcd091c640 1 -- 172.21.15.92:0/2656469528 shutdown_connections 2024-04-22T07:04:10.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.267+0000 7f222e1c8640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f220c06c940 0x7f220c06ee00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2224034000 tx=0x7f2224009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f885f6ba640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8858073c10 con 0x7f88580c1c40 2024-04-22T07:04:10.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f885f6ba640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f88580c2020 con 0x7f88581f1590 2024-04-22T07:04:10.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f885f6ba640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8858073d90 con 0x7f885806e260 2024-04-22T07:04:10.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.266+0000 7f885d456640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f885806e260 0x7f88581fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.267+0000 7f885dc57640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88581f1590 0x7f88581f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.267+0000 7f885dc57640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88581f1590 0x7f88581f2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:53718/0 (socket says 172.21.15.92:53718) 2024-04-22T07:04:10.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.267+0000 7f885dc57640 1 -- 172.21.15.92:0/307917140 learned_addr learned my addr 172.21.15.92:0/307917140 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.267+0000 7f885cc55640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88580c1c40 0x7f8858161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.267+0000 7f885dc57640 1 -- 172.21.15.92:0/307917140 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f885806e260 msgr2=0x7f88581fe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.267+0000 7efcd091c640 1 -- 172.21.15.92:0/2656469528 wait complete. 2024-04-22T07:04:10.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.267+0000 7f221effd640 1 -- 172.21.15.92:0/1512330089 <== osd.1 v2:172.21.15.92:6816/2805470585 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f220c077690 con 0x7f220c074bc0 2024-04-22T07:04:10.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.268+0000 7f885dc57640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f885806e260 0x7f88581fe1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.268+0000 7f885dc57640 1 -- 172.21.15.92:0/307917140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88580c1c40 msgr2=0x7f8858161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.268+0000 7f885dc57640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88580c1c40 0x7f8858161040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.268+0000 7f885dc57640 1 -- 172.21.15.92:0/307917140 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88581f1ce0 con 0x7f88581f1590 2024-04-22T07:04:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.268+0000 7f885cc55640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88580c1c40 0x7f8858161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.268+0000 7f885dc57640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88581f1590 0x7f88581f2d50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f884800c420 tx=0x7f884800c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.268+0000 7f885d456640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f885806e260 0x7f88581fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.268+0000 7f884dffb640 1 -- 172.21.15.92:0/307917140 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88480052d0 con 0x7f88581f1590 2024-04-22T07:04:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.269+0000 7f884dffb640 1 -- 172.21.15.92:0/307917140 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8848005470 con 0x7f88581f1590 2024-04-22T07:04:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.270+0000 7f884dffb640 1 -- 172.21.15.92:0/307917140 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848007260 con 0x7f88581f1590 2024-04-22T07:04:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.269+0000 7efcd091c640 1 Processor -- start 2024-04-22T07:04:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.269+0000 7efcd091c640 1 -- start start 2024-04-22T07:04:10.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.270+0000 7f885f6ba640 1 -- 172.21.15.92:0/307917140 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88581f1f80 con 0x7f88581f1590 2024-04-22T07:04:10.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.270+0000 7efcd091c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc4045720 0x7efcc40db0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.270+0000 7efcd091c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40d5090 0x7efcc4035250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.270+0000 7efccb7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc4045720 0x7efcc40db0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.270+0000 7efccb7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc4045720 0x7efcc40db0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52354/0 (socket says 172.21.15.92:52354) 2024-04-22T07:04:10.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.270+0000 7f885f6ba640 1 -- 172.21.15.92:0/307917140 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88580cc960 con 0x7f88581f1590 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.271+0000 7efcd091c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcc40de3b0 0x7efcc4035790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.271+0000 7efcd091c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efcc404cb50 con 0x7efcc40d5090 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.271+0000 7efcd091c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efcc40d5470 con 0x7efcc40de3b0 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.271+0000 7efcd091c640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efcc404ccd0 con 0x7efcc4045720 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.271+0000 7efccaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40d5090 0x7efcc4035250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.271+0000 7efccb7fe640 1 -- 172.21.15.92:0/2768328176 learned_addr learned my addr 172.21.15.92:0/2768328176 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.271+0000 7efccbfff640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcc40de3b0 0x7efcc4035790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.272+0000 7efccbfff640 1 -- 172.21.15.92:0/2768328176 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc4045720 msgr2=0x7efcc40db0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.272+0000 7efccbfff640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc4045720 0x7efcc40db0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.272+0000 7efccbfff640 1 -- 172.21.15.92:0/2768328176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40d5090 msgr2=0x7efcc4035250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.272+0000 7efccbfff640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40d5090 0x7efcc4035250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.272+0000 7efccbfff640 1 -- 172.21.15.92:0/2768328176 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc404ff70 con 0x7efcc40de3b0 2024-04-22T07:04:10.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.272+0000 7efccb7fe640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc4045720 0x7efcc40db0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:10.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.272+0000 7efccaffd640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40d5090 0x7efcc4035250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:10.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.272+0000 7efccbfff640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcc40de3b0 0x7efcc4035790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efcb8033000 tx=0x7efcb80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.272+0000 7f885f6ba640 1 -- 172.21.15.92:0/307917140 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f885808cc90 con 0x7f88581f1590 2024-04-22T07:04:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.273+0000 7f884dffb640 1 -- 172.21.15.92:0/307917140 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8848007400 con 0x7f88581f1590 2024-04-22T07:04:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.273+0000 7f884dffb640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f883c06c9f0 0x7f883c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.273+0000 7f884dffb640 1 -- 172.21.15.92:0/307917140 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f884808ae30 con 0x7f88581f1590 2024-04-22T07:04:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.273+0000 7f885d456640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f883c06c9f0 0x7f883c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.273+0000 7f884dffb640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.145:6824/97436481,v1:172.21.15.145:6825/97436481] conn(0x7f883c074c70 0x7f883c077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.273+0000 7f884dffb640 1 -- 172.21.15.92:0/307917140 --> [v2:172.21.15.145:6824/97436481,v1:172.21.15.145:6825/97436481] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8848006e70 con 0x7f883c074c70 2024-04-22T07:04:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.273+0000 7f884dffb640 1 -- 172.21.15.92:0/307917140 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8848090420 con 0x7f88581f1590 2024-04-22T07:04:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.274+0000 7f885d456640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f883c06c9f0 0x7f883c06eeb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f885403a000 tx=0x7f8854034000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.274+0000 7f885cc55640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.145:6824/97436481,v1:172.21.15.145:6825/97436481] conn(0x7f883c074c70 0x7f883c077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.274+0000 7efcb7fff640 1 -- 172.21.15.92:0/2768328176 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8037c80 con 0x7efcc40de3b0 2024-04-22T07:04:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.274+0000 7efcb7fff640 1 -- 172.21.15.92:0/2768328176 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb8047070 con 0x7efcc40de3b0 2024-04-22T07:04:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.274+0000 7efcd091c640 1 -- 172.21.15.92:0/2768328176 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc4050250 con 0x7efcc40de3b0 2024-04-22T07:04:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.274+0000 7efcb7fff640 1 -- 172.21.15.92:0/2768328176 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8042680 con 0x7efcc40de3b0 2024-04-22T07:04:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.274+0000 7efcd091c640 1 -- 172.21.15.92:0/2768328176 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc40d1940 con 0x7efcc40de3b0 2024-04-22T07:04:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.275+0000 7f885cc55640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.145:6824/97436481,v1:172.21.15.145:6825/97436481] conn(0x7f883c074c70 0x7f883c077090 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.275+0000 7efcd091c640 1 -- 172.21.15.92:0/2768328176 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efcac001030 con 0x7efcc40de3b0 2024-04-22T07:04:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.276+0000 7f884dffb640 1 -- 172.21.15.92:0/307917140 <== osd.7 v2:172.21.15.145:6824/97436481 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8848006e70 con 0x7f883c074c70 2024-04-22T07:04:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.276+0000 7efcb7fff640 1 -- 172.21.15.92:0/2768328176 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcb8040050 con 0x7efcc40de3b0 2024-04-22T07:04:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.277+0000 7efcb7fff640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efcb006c9c0 0x7efcb006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.277+0000 7efccb7fe640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efcb006c9c0 0x7efcb006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.278+0000 7efccb7fe640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efcb006c9c0 0x7efcb006ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efccc076410 tx=0x7efccc06e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.278+0000 7efcb7fff640 1 -- 172.21.15.92:0/2768328176 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efcb80ae490 con 0x7efcc40de3b0 2024-04-22T07:04:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.278+0000 7efcb7fff640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.145:6816/290698449,v1:172.21.15.145:6817/290698449] conn(0x7efcb0074b60 0x7efcb0076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.278+0000 7efcb7fff640 1 -- 172.21.15.92:0/2768328176 --> [v2:172.21.15.145:6816/290698449,v1:172.21.15.145:6817/290698449] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efcb800af30 con 0x7efcb0074b60 2024-04-22T07:04:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.279+0000 7efcb7fff640 1 -- 172.21.15.92:0/2768328176 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7efcb80b5050 con 0x7efcc40de3b0 2024-04-22T07:04:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.279+0000 7efccaffd640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.145:6816/290698449,v1:172.21.15.145:6817/290698449] conn(0x7efcb0074b60 0x7efcb0076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.279+0000 7efccaffd640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.145:6816/290698449,v1:172.21.15.145:6817/290698449] conn(0x7efcb0074b60 0x7efcb0076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.280+0000 7efcb7fff640 1 -- 172.21.15.92:0/2768328176 <== osd.6 v2:172.21.15.145:6816/290698449 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efcb800af30 con 0x7efcb0074b60 2024-04-22T07:04:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.290+0000 7f309cd4a640 1 -- 172.21.15.92:0/2392829313 --> [v2:172.21.15.92:6800/24674040,v1:172.21.15.92:6801/24674040] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f307c002dd0 con 0x7f3078074bf0 2024-04-22T07:04:10.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.290+0000 7f3086ffd640 1 -- 172.21.15.92:0/2392829313 <== osd.3 v2:172.21.15.92:6800/24674040 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f307c002dd0 con 0x7f3078074bf0 2024-04-22T07:04:10.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.291+0000 7f3084ff9640 1 -- 172.21.15.92:0/2392829313 >> [v2:172.21.15.92:6800/24674040,v1:172.21.15.92:6801/24674040] conn(0x7f3078074bf0 msgr2=0x7f3078077010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.291+0000 7f3084ff9640 1 --2- 172.21.15.92:0/2392829313 >> [v2:172.21.15.92:6800/24674040,v1:172.21.15.92:6801/24674040] conn(0x7f3078074bf0 0x7f3078077010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.291+0000 7f3084ff9640 1 -- 172.21.15.92:0/2392829313 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f307806c9f0 msgr2=0x7f307806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.292+0000 7f3084ff9640 1 --2- 172.21.15.92:0/2392829313 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f307806c9f0 0x7f307806eeb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f30980c9ba0 tx=0x7f308c008230 comp rx=0 tx=0).stop 2024-04-22T07:04:10.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.292+0000 7f3084ff9640 1 -- 172.21.15.92:0/2392829313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30980c4cd0 msgr2=0x7f30980c5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.292+0000 7f3084ff9640 1 --2- 172.21.15.92:0/2392829313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30980c4cd0 0x7f30980c5150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f309806ce50 tx=0x7f3090011580 comp rx=0 tx=0).stop 2024-04-22T07:04:10.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.293+0000 7f3096d9d640 1 -- 172.21.15.92:0/2392829313 reap_dead start 2024-04-22T07:04:10.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.293+0000 7f3084ff9640 1 -- 172.21.15.92:0/2392829313 shutdown_connections 2024-04-22T07:04:10.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.293+0000 7f3084ff9640 1 -- 172.21.15.92:0/2392829313 >> 172.21.15.92:0/2392829313 conn(0x7f309806bfd0 msgr2=0x7f3098163350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.293+0000 7f3084ff9640 1 -- 172.21.15.92:0/2392829313 shutdown_connections 2024-04-22T07:04:10.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.293+0000 7f3084ff9640 1 -- 172.21.15.92:0/2392829313 wait complete. 2024-04-22T07:04:10.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.294+0000 7ff43109a640 1 -- 172.21.15.92:0/3482513486 --> [v2:172.21.15.92:6824/3408935700,v1:172.21.15.92:6825/3408935700] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff4241796a0 con 0x7ff418074c70 2024-04-22T07:04:10.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.295+0000 7ff4137fe640 1 -- 172.21.15.92:0/3482513486 <== osd.0 v2:172.21.15.92:6824/3408935700 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff4241796a0 con 0x7ff418074c70 2024-04-22T07:04:10.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.295+0000 7ff43109a640 1 -- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:6824/3408935700,v1:172.21.15.92:6825/3408935700] conn(0x7ff418074c70 msgr2=0x7ff418077090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.295+0000 7ff43109a640 1 --2- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:6824/3408935700,v1:172.21.15.92:6825/3408935700] conn(0x7ff418074c70 0x7ff418077090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.295+0000 7ff43109a640 1 -- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff41806c9f0 msgr2=0x7ff41806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.295+0000 7ff43109a640 1 --2- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff41806c9f0 0x7ff41806eeb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff42c050f50 tx=0x7ff42c075de0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.295+0000 7ff43109a640 1 -- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff42404ba60 msgr2=0x7ff424179a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.295+0000 7ff43109a640 1 --2- 172.21.15.92:0/3482513486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff42404ba60 0x7ff424179a70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff42000c7f0 tx=0x7ff42000cbf0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.295+0000 7ff42b59e640 1 -- 172.21.15.92:0/3482513486 reap_dead start 2024-04-22T07:04:10.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.297+0000 7ff43109a640 1 -- 172.21.15.92:0/3482513486 shutdown_connections 2024-04-22T07:04:10.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.297+0000 7ff43109a640 1 -- 172.21.15.92:0/3482513486 >> 172.21.15.92:0/3482513486 conn(0x7ff424050180 msgr2=0x7ff42404f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.297+0000 7ff43109a640 1 -- 172.21.15.92:0/3482513486 shutdown_connections 2024-04-22T07:04:10.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.297+0000 7ff43109a640 1 -- 172.21.15.92:0/3482513486 wait complete. 2024-04-22T07:04:10.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.300+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 --> [v2:172.21.15.145:6800/2981429006,v1:172.21.15.145:6801/2981429006] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb0c4002dd0 con 0x7fb0c0074b60 2024-04-22T07:04:10.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.300+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 --> [v2:172.21.15.92:6808/3594806740,v1:172.21.15.92:6809/3594806740] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc308005c60 con 0x7fc308001650 2024-04-22T07:04:10.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.300+0000 7fb0ceffd640 1 -- 172.21.15.92:0/1745708359 <== osd.5 v2:172.21.15.145:6800/2981429006 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb0c4002dd0 con 0x7fb0c0074b60 2024-04-22T07:04:10.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.300+0000 7f222f1ca640 1 -- 172.21.15.92:0/1512330089 --> [v2:172.21.15.92:6816/2805470585,v1:172.21.15.92:6817/2805470585] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f22200057e0 con 0x7f220c074bc0 2024-04-22T07:04:10.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fc31dffb640 1 -- 172.21.15.92:0/2492677980 <== osd.2 v2:172.21.15.92:6808/3594806740 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc308005c60 con 0x7fc308001650 2024-04-22T07:04:10.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7f2da6c06640 1 -- 172.21.15.92:0/2832159377 --> [v2:172.21.15.145:6804/3070790698,v1:172.21.15.145:6806/3070790698] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d80002dd0 con 0x7f2d8c074ae0 2024-04-22T07:04:10.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7f221effd640 1 -- 172.21.15.92:0/1512330089 <== osd.1 v2:172.21.15.92:6816/2805470585 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f22200057e0 con 0x7f220c074bc0 2024-04-22T07:04:10.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 >> [v2:172.21.15.145:6800/2981429006,v1:172.21.15.145:6801/2981429006] conn(0x7fb0c0074b60 msgr2=0x7fb0c0076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fb0e4bb1640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.145:6800/2981429006,v1:172.21.15.145:6801/2981429006] conn(0x7fb0c0074b60 0x7fb0c0076f80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:6808/3594806740,v1:172.21.15.92:6809/3594806740] conn(0x7fc308001650 msgr2=0x7fc308003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fc32f396640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:6808/3594806740,v1:172.21.15.92:6809/3594806740] conn(0x7fc308001650 0x7fc308003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7f2d9d7fa640 1 -- 172.21.15.92:0/2832159377 <== osd.4 v2:172.21.15.145:6804/3070790698 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2d80002dd0 con 0x7f2d8c074ae0 2024-04-22T07:04:10.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb0c006c940 msgr2=0x7fb0c006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7f222f1ca640 1 -- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:6816/2805470585,v1:172.21.15.92:6817/2805470585] conn(0x7f220c074bc0 msgr2=0x7f220c076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc30c06ccf0 msgr2=0x7fc30c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fc32f396640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc30c06ccf0 0x7fc30c06f1b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc324001820 tx=0x7fc324009040 comp rx=0 tx=0).stop 2024-04-22T07:04:10.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3200dc700 msgr2=0x7fc3200cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fc32f396640 1 --2- 172.21.15.92:0/2492677980 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3200dc700 0x7fc3200cd5a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc328050a90 tx=0x7fc3280ab070 comp rx=0 tx=0).stop 2024-04-22T07:04:10.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fc32d933640 1 -- 172.21.15.92:0/2492677980 reap_dead start 2024-04-22T07:04:10.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7f222f1ca640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:6816/2805470585,v1:172.21.15.92:6817/2805470585] conn(0x7f220c074bc0 0x7f220c076fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 shutdown_connections 2024-04-22T07:04:10.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 >> 172.21.15.92:0/2492677980 conn(0x7fc32004d4b0 msgr2=0x7fc32004b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fb0e4bb1640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb0c006c940 0x7fb0c006ee00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00099f0 tx=0x7fb0d0008040 comp rx=0 tx=0).stop 2024-04-22T07:04:10.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d80dd400 msgr2=0x7fb0d80e6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.301+0000 7fb0e4bb1640 1 --2- 172.21.15.92:0/1745708359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d80dd400 0x7fb0d80e6810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400c450 tx=0x7fb0d400c910 comp rx=0 tx=0).stop 2024-04-22T07:04:10.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7fb0ded9d640 1 -- 172.21.15.92:0/1745708359 reap_dead start 2024-04-22T07:04:10.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 shutdown_connections 2024-04-22T07:04:10.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 >> 172.21.15.92:0/1745708359 conn(0x7fb0d800e160 msgr2=0x7fb0d804d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7f222f1ca640 1 -- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f220c06c940 msgr2=0x7f220c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7f222f1ca640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f220c06c940 0x7f220c06ee00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2224034000 tx=0x7f2224009030 comp rx=0 tx=0).stop 2024-04-22T07:04:10.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7f222f1ca640 1 -- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2220173fa0 msgr2=0x7f22200cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7f222f1ca640 1 --2- 172.21.15.92:0/1512330089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2220173fa0 0x7f22200cf030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f221800ca40 tx=0x7f221800cf00 comp rx=0 tx=0).stop 2024-04-22T07:04:10.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 shutdown_connections 2024-04-22T07:04:10.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7fb0e4bb1640 1 -- 172.21.15.92:0/1745708359 wait complete. 2024-04-22T07:04:10.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7f222e9c9640 1 -- 172.21.15.92:0/1512330089 reap_dead start 2024-04-22T07:04:10.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7f222f1ca640 1 -- 172.21.15.92:0/1512330089 shutdown_connections 2024-04-22T07:04:10.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7f222f1ca640 1 -- 172.21.15.92:0/1512330089 >> 172.21.15.92:0/1512330089 conn(0x7f222000e6a0 msgr2=0x7f22200d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 shutdown_connections 2024-04-22T07:04:10.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7fc32f396640 1 -- 172.21.15.92:0/2492677980 wait complete. 2024-04-22T07:04:10.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7f222f1ca640 1 -- 172.21.15.92:0/1512330089 shutdown_connections 2024-04-22T07:04:10.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7f2d86ffd640 1 -- 172.21.15.92:0/2832159377 >> [v2:172.21.15.145:6804/3070790698,v1:172.21.15.145:6806/3070790698] conn(0x7f2d8c074ae0 msgr2=0x7f2d8c076f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.302+0000 7f2d86ffd640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.145:6804/3070790698,v1:172.21.15.145:6806/3070790698] conn(0x7f2d8c074ae0 0x7f2d8c076f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.303+0000 7f222f1ca640 1 -- 172.21.15.92:0/1512330089 wait complete. 2024-04-22T07:04:10.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.303+0000 7f2d86ffd640 1 -- 172.21.15.92:0/2832159377 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f2d8c06c940 msgr2=0x7f2d8c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.303+0000 7f2d86ffd640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f2d8c06c940 0x7f2d8c06ee00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d98036000 tx=0x7f2d98030000 comp rx=0 tx=0).stop 2024-04-22T07:04:10.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.303+0000 7f2d86ffd640 1 -- 172.21.15.92:0/2832159377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00c7630 msgr2=0x7f2da015d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.303+0000 7f2d86ffd640 1 --2- 172.21.15.92:0/2832159377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00c7630 0x7f2da015d400 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400c410 tx=0x7f2d9400c8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.303+0000 7f2da51a3640 1 -- 172.21.15.92:0/2832159377 reap_dead start 2024-04-22T07:04:10.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.303+0000 7f2d86ffd640 1 -- 172.21.15.92:0/2832159377 shutdown_connections 2024-04-22T07:04:10.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.303+0000 7f2d86ffd640 1 -- 172.21.15.92:0/2832159377 >> 172.21.15.92:0/2832159377 conn(0x7f2da00cc220 msgr2=0x7f2da0153990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.303+0000 7f2d86ffd640 1 -- 172.21.15.92:0/2832159377 shutdown_connections 2024-04-22T07:04:10.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.303+0000 7f2d86ffd640 1 -- 172.21.15.92:0/2832159377 wait complete. 2024-04-22T07:04:10.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.307+0000 7f885f6ba640 1 -- 172.21.15.92:0/307917140 --> [v2:172.21.15.145:6824/97436481,v1:172.21.15.145:6825/97436481] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f885804fdc0 con 0x7f883c074c70 2024-04-22T07:04:10.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.308+0000 7f884dffb640 1 -- 172.21.15.92:0/307917140 <== osd.7 v2:172.21.15.145:6824/97436481 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f885804fdc0 con 0x7f883c074c70 2024-04-22T07:04:10.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.309+0000 7f883b7fe640 1 -- 172.21.15.92:0/307917140 >> [v2:172.21.15.145:6824/97436481,v1:172.21.15.145:6825/97436481] conn(0x7f883c074c70 msgr2=0x7f883c077090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.309+0000 7f883b7fe640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.145:6824/97436481,v1:172.21.15.145:6825/97436481] conn(0x7f883c074c70 0x7f883c077090 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.309+0000 7f883b7fe640 1 -- 172.21.15.92:0/307917140 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f883c06c9f0 msgr2=0x7f883c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.309+0000 7f883b7fe640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f883c06c9f0 0x7f883c06eeb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f885403a000 tx=0x7f8854034000 comp rx=0 tx=0).stop 2024-04-22T07:04:10.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.309+0000 7f883b7fe640 1 -- 172.21.15.92:0/307917140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88581f1590 msgr2=0x7f88581f2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.309+0000 7f883b7fe640 1 --2- 172.21.15.92:0/307917140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88581f1590 0x7f88581f2d50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f884800c420 tx=0x7f884800c8e0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.309+0000 7f885dc57640 1 -- 172.21.15.92:0/307917140 reap_dead start 2024-04-22T07:04:10.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.309+0000 7f883b7fe640 1 -- 172.21.15.92:0/307917140 shutdown_connections 2024-04-22T07:04:10.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.309+0000 7f883b7fe640 1 -- 172.21.15.92:0/307917140 >> 172.21.15.92:0/307917140 conn(0x7f885806bfd0 msgr2=0x7f88580b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.309+0000 7f883b7fe640 1 -- 172.21.15.92:0/307917140 shutdown_connections 2024-04-22T07:04:10.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.310+0000 7f883b7fe640 1 -- 172.21.15.92:0/307917140 wait complete. 2024-04-22T07:04:10.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.311+0000 7efcd091c640 1 -- 172.21.15.92:0/2768328176 --> [v2:172.21.15.145:6816/290698449,v1:172.21.15.145:6817/290698449] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efcac002dd0 con 0x7efcb0074b60 2024-04-22T07:04:10.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.312+0000 7efcb7fff640 1 -- 172.21.15.92:0/2768328176 <== osd.6 v2:172.21.15.145:6816/290698449 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efcac002dd0 con 0x7efcb0074b60 2024-04-22T07:04:10.370 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:10.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-22T07:04:10.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.313+0000 7efcd091c640 1 -- 172.21.15.92:0/2768328176 >> [v2:172.21.15.145:6816/290698449,v1:172.21.15.145:6817/290698449] conn(0x7efcb0074b60 msgr2=0x7efcb0076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.313+0000 7efcd091c640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.145:6816/290698449,v1:172.21.15.145:6817/290698449] conn(0x7efcb0074b60 0x7efcb0076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.313+0000 7efcd091c640 1 -- 172.21.15.92:0/2768328176 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efcb006c9c0 msgr2=0x7efcb006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.313+0000 7efcd091c640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efcb006c9c0 0x7efcb006ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efccc076410 tx=0x7efccc06e040 comp rx=0 tx=0).stop 2024-04-22T07:04:10.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.313+0000 7efcd091c640 1 -- 172.21.15.92:0/2768328176 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcc40de3b0 msgr2=0x7efcc4035790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.313+0000 7efcd091c640 1 --2- 172.21.15.92:0/2768328176 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcc40de3b0 0x7efcc4035790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efcb8033000 tx=0x7efcb80046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.313+0000 7efccbfff640 1 -- 172.21.15.92:0/2768328176 reap_dead start 2024-04-22T07:04:10.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.313+0000 7efcd091c640 1 -- 172.21.15.92:0/2768328176 shutdown_connections 2024-04-22T07:04:10.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.313+0000 7efcd091c640 1 -- 172.21.15.92:0/2768328176 >> 172.21.15.92:0/2768328176 conn(0x7efcc40505d0 msgr2=0x7efcc40d3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.314+0000 7efcd091c640 1 -- 172.21.15.92:0/2768328176 shutdown_connections 2024-04-22T07:04:10.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.314+0000 7efcd091c640 1 -- 172.21.15.92:0/2768328176 wait complete. 2024-04-22T07:04:10.373 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:10.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-22T07:04:10.374 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:10.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-22T07:04:10.375 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2024-04-22T07:04:10.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-22T07:04:10.376 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2024-04-22T07:04:10.376 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2024-04-22T07:04:10.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-22T07:04:10.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-22T07:04:10.378 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2024-04-22T07:04:10.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-22T07:04:10.379 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:10.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-22T07:04:10.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.580+0000 7f463c0d2640 1 Processor -- start 2024-04-22T07:04:10.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.580+0000 7f3ef6723640 1 Processor -- start 2024-04-22T07:04:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.580+0000 7f463c0d2640 1 -- start start 2024-04-22T07:04:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.581+0000 7ff7b04c1640 1 Processor -- start 2024-04-22T07:04:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.581+0000 7f7e34411640 1 Processor -- start 2024-04-22T07:04:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.581+0000 7ff7b04c1640 1 -- start start 2024-04-22T07:04:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.581+0000 7f463c0d2640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f46340c8020 0x7f463406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7f463a66f640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f46340b1560 0x7f4634161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46652/0 (socket says 172.21.15.92:46652) 2024-04-22T07:04:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7f463a66f640 1 -- 172.21.15.92:0/4131248438 learned_addr learned my addr 172.21.15.92:0/4131248438 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7f463966d640 1 --2- 172.21.15.92:0/4131248438 >> v2:172.21.15.145:3300/0 conn(0x7f46340c8020 0x7f463406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7f463c0d2640 1 -- 172.21.15.92:0/4131248438 --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4634156e80 con 0x7f46340b1560 2024-04-22T07:04:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7f463c0d2640 1 -- 172.21.15.92:0/4131248438 --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4634157160 con 0x7f46340c7c40 2024-04-22T07:04:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7f463c0d2640 1 -- 172.21.15.92:0/4131248438 --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f463406d340 con 0x7f46340c8020 2024-04-22T07:04:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.580+0000 7f3ef6723640 1 -- start start 2024-04-22T07:04:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.581+0000 7f3ef6723640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3ef00c7c40 0x7f3ef00b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.581+0000 7f3ef6723640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ef006dee0 0x7f3ef00b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.581+0000 7f3ef6723640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef0156e80 con 0x7f3ef00c8020 2024-04-22T07:04:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.581+0000 7f3ef6723640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef006d340 con 0x7f3ef006dee0 2024-04-22T07:04:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.581+0000 7f3ef6723640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef0157160 con 0x7f3ef00c7c40 2024-04-22T07:04:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7f3ef4cc0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ef006dee0 0x7f3ef00b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7f3ef4cc0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ef006dee0 0x7f3ef00b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36158/0 (socket says 172.21.15.92:36158) 2024-04-22T07:04:10.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7f3ef4cc0640 1 -- 172.21.15.92:0/3603465456 learned_addr learned my addr 172.21.15.92:0/3603465456 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7f3eef7fe640 1 --1- 172.21.15.92:0/3603465456 >> v1:172.21.15.92:6789/0 conn(0x7f3ef00c8020 0x7f3ef006d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46636/0 (socket says 172.21.15.92:46636) 2024-04-22T07:04:10.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7efcd27cf640 1 Processor -- start 2024-04-22T07:04:10.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7ff7b04c1640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff7a00dc570 0x7ff7a0007a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7ff7b04c1640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7a004c630 con 0x7ff7a00dba40 2024-04-22T07:04:10.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7ff7b04c1640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7a0046440 con 0x7ff7a00d5e30 2024-04-22T07:04:10.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7ff7b04c1640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff7a00d6210 con 0x7ff7a00dc570 2024-04-22T07:04:10.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7efcd27cf640 1 -- start start 2024-04-22T07:04:10.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2994683772 0 0) 0x7f4634157160 con 0x7f46340c7c40 2024-04-22T07:04:10.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7ff7ae25d640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7ff7a00d5e30 0x7ff7a0047c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:55192/0 (socket says 172.21.15.92:55192) 2024-04-22T07:04:10.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7ff7ae25d640 1 -- 172.21.15.92:0/893299968 learned_addr learned my addr 172.21.15.92:0/893299968 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:10.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1084036153 0 0) 0x7ff7a0046440 con 0x7ff7a00d5e30 2024-04-22T07:04:10.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f9746d92640 1 Processor -- start 2024-04-22T07:04:10.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f3ef4cc0640 1 -- 172.21.15.92:0/3603465456 >> v2:172.21.15.145:3300/0 conn(0x7f3ef00c7c40 msgr2=0x7f3ef00b0e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:04:10.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f3ef4cc0640 1 --2- 172.21.15.92:0/3603465456 >> v2:172.21.15.145:3300/0 conn(0x7f3ef00c7c40 0x7f3ef00b0e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7ff7aea5e640 1 --2- 172.21.15.92:0/893299968 >> v2:172.21.15.145:3300/0 conn(0x7ff7a00dc570 0x7ff7a0007a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff794003520 con 0x7ff7a00d5e30 2024-04-22T07:04:10.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.581+0000 7f7e34411640 1 -- start start 2024-04-22T07:04:10.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e34411640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e24050aa0 0x7f7e240e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e34411640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2404a790 0x7f7e2404ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f3eeeffd640 1 -- 172.21.15.92:0/3603465456 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1422462420 0 0) 0x7f3ef0156e80 con 0x7f3ef00c8020 2024-04-22T07:04:10.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f3ef4cc0640 1 -- 172.21.15.92:0/3603465456 >> v1:172.21.15.92:6789/0 conn(0x7f3ef00c8020 legacy=0x7f3ef006d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e34411640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2404f640 con 0x7f7e240dd6d0 2024-04-22T07:04:10.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.582+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4620003520 con 0x7f46340c7c40 2024-04-22T07:04:10.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 <== mon.1 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3752509341 0 0) 0x7f4620003520 con 0x7f46340c7c40 2024-04-22T07:04:10.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 >> v2:172.21.15.145:3300/0 conn(0x7f46340c8020 msgr2=0x7f463406d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 --2- 172.21.15.92:0/4131248438 >> v2:172.21.15.145:3300/0 conn(0x7f46340c8020 0x7f463406d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 >> v1:172.21.15.92:6789/0 conn(0x7f46340b1560 legacy=0x7f4634161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4634157440 con 0x7f46340c7c40 2024-04-22T07:04:10.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 <== mon.1 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f4630001670 con 0x7f46340c7c40 2024-04-22T07:04:10.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 >> v1:172.21.15.145:6789/0 conn(0x7f46340c7c40 legacy=0x7f46340b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 --2- 172.21.15.92:0/4131248438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46200039b0 0x7f4620003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 --2- 172.21.15.92:0/4131248438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4620004800 0x7f4620004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 --2- 172.21.15.92:0/4131248438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46200051c0 0x7f462000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f463406d340 con 0x7f46200051c0 2024-04-22T07:04:10.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f46200148a0 con 0x7f4620004800 2024-04-22T07:04:10.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f462000bfb0 con 0x7f46200039b0 2024-04-22T07:04:10.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f3ef4cc0640 1 -- 172.21.15.92:0/3603465456 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef0157440 con 0x7f3ef006dee0 2024-04-22T07:04:10.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 <== mon.1 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3562414040 0 0) 0x7ff794003520 con 0x7ff7a00d5e30 2024-04-22T07:04:10.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 >> v2:172.21.15.145:3300/0 conn(0x7ff7a00dc570 msgr2=0x7ff7a0007a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7ff7ad25b640 1 --2- 172.21.15.92:0/893299968 >> v2:172.21.15.145:3300/0 conn(0x7ff7a00dc570 0x7ff7a0007a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 >> v1:172.21.15.92:6789/0 conn(0x7ff7a00dba40 legacy=0x7ff7a00dbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a00e2390 con 0x7ff7a00d5e30 2024-04-22T07:04:10.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 <== mon.1 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7ff7a80743a0 con 0x7ff7a00d5e30 2024-04-22T07:04:10.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 >> v1:172.21.15.145:6789/0 conn(0x7ff7a00d5e30 legacy=0x7ff7a0047c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7ff7ad25b640 1 --2- 172.21.15.92:0/893299968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7940039b0 0x7ff794003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7ff7ad25b640 1 --2- 172.21.15.92:0/893299968 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff794004800 0x7ff794004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7ff7ad25b640 1 --2- 172.21.15.92:0/893299968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7940051c0 0x7ff79400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7ff7ada5c640 1 --2- 172.21.15.92:0/893299968 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff794004800 0x7ff794004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7ff7aea5e640 1 --2- 172.21.15.92:0/893299968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7940051c0 0x7ff79400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a00d6210 con 0x7ff7940051c0 2024-04-22T07:04:10.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff79400c010 con 0x7ff7940039b0 2024-04-22T07:04:10.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff79400c210 con 0x7ff794004800 2024-04-22T07:04:10.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f9746d92640 1 -- start start 2024-04-22T07:04:10.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e34411640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f7e240e4140 con 0x7f7e24050aa0 2024-04-22T07:04:10.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e34411640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7e240496c0 con 0x7f7e2404a790 2024-04-22T07:04:10.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e329ae640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2404a790 0x7f7e2404ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e329ae640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2404a790 0x7f7e2404ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52358/0 (socket says 172.21.15.92:52358) 2024-04-22T07:04:10.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e329ae640 1 -- 172.21.15.92:0/2744678288 learned_addr learned my addr 172.21.15.92:0/2744678288 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e319ac640 1 --2- 172.21.15.92:0/2744678288 >> v2:172.21.15.92:3300/0 conn(0x7f7e24050aa0 0x7f7e240e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e329ae640 1 -- 172.21.15.92:0/2744678288 >> v2:172.21.15.92:3300/0 conn(0x7f7e24050aa0 msgr2=0x7f7e240e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e329ae640 1 --2- 172.21.15.92:0/2744678288 >> v2:172.21.15.92:3300/0 conn(0x7f7e24050aa0 0x7f7e240e3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e329ae640 1 -- 172.21.15.92:0/2744678288 >> v1:172.21.15.145:6789/0 conn(0x7f7e240dd6d0 legacy=0x7f7e2404ff60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:04:10.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7f7e329ae640 1 -- 172.21.15.92:0/2744678288 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e240499a0 con 0x7f7e2404a790 2024-04-22T07:04:10.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7efcd27cf640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efcc40d9240 0x7efcc4039e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7efcd27cf640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40e0480 0x7efcc404ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7efcd27cf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efcc40dfa90 0x7efcc40071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7efcd27cf640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7efcc4050500 con 0x7efcc40dfa90 2024-04-22T07:04:10.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7efcd27cf640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7efcc403a4a0 con 0x7efcc40d9240 2024-04-22T07:04:10.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.583+0000 7efcd27cf640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efcc404ef20 con 0x7efcc40e0480 2024-04-22T07:04:10.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7efcd1fce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efcc40dfa90 0x7efcc40071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7efcd1fce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efcc40dfa90 0x7efcc40071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36182/0 (socket says 172.21.15.92:36182) 2024-04-22T07:04:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7efcd1fce640 1 -- 172.21.15.92:0/1199294852 learned_addr learned my addr 172.21.15.92:0/1199294852 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7efcd17cd640 1 --2- 172.21.15.92:0/1199294852 >> v2:172.21.15.145:3300/0 conn(0x7efcc40d9240 0x7efcc4039e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7efcd0fcc640 1 --2- 172.21.15.92:0/1199294852 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40e0480 0x7efcc404ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7f463966d640 1 --2- 172.21.15.92:0/4131248438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4620004800 0x7f4620004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7f9746d92640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9740152ab0 0x7f97400c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7f9746d92640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f97400c4f50 0x7f9740158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7f9746d92640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f974015be80 con 0x7f97400c37d0 2024-04-22T07:04:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7f9746d92640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f974015c120 con 0x7f97400c4f50 2024-04-22T07:04:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7f9746d92640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f974015c400 con 0x7f9740152ab0 2024-04-22T07:04:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7f463a66f640 1 --2- 172.21.15.92:0/4131248438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46200051c0 0x7f462000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7f974558f640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f97400c37d0 0x7f97400713f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:55206/0 (socket says 172.21.15.92:55206) 2024-04-22T07:04:10.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7f974558f640 1 -- 172.21.15.92:0/980957643 learned_addr learned my addr 172.21.15.92:0/980957643 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:10.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7f9746591640 1 --2- 172.21.15.92:0/980957643 >> v2:172.21.15.92:3300/0 conn(0x7f97400c4f50 0x7f9740158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f4639e6e640 1 --2- 172.21.15.92:0/4131248438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46200039b0 0x7f4620003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7f9745d90640 1 --2- 172.21.15.92:0/980957643 >> v2:172.21.15.145:3300/0 conn(0x7f9740152ab0 0x7f97400c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7f7e329ae640 1 --2- 172.21.15.92:0/2744678288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2404a790 0x7f7e2404ac20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c02db50 tx=0x7f7e2c0a7b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=805224de209e9157 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f3ef4cc0640 1 --2- 172.21.15.92:0/3603465456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef006dee0 0x7f3ef00b1390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00700a0 tx=0x7f3ee402df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5c2c16c7370104 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7efcd1fce640 1 -- 172.21.15.92:0/1199294852 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40e0480 msgr2=0x7efcc404ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7efcd1fce640 1 --2- 172.21.15.92:0/1199294852 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40e0480 0x7efcc404ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7efcd1fce640 1 -- 172.21.15.92:0/1199294852 >> v2:172.21.15.145:3300/0 conn(0x7efcc40d9240 msgr2=0x7efcc4039e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7efcd1fce640 1 --2- 172.21.15.92:0/1199294852 >> v2:172.21.15.145:3300/0 conn(0x7efcc40d9240 0x7efcc4039e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7efcd1fce640 1 -- 172.21.15.92:0/1199294852 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc404f200 con 0x7efcc40dfa90 2024-04-22T07:04:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f463966d640 1 -- 172.21.15.92:0/4131248438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46200039b0 msgr2=0x7f4620003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f463966d640 1 --2- 172.21.15.92:0/4131248438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46200039b0 0x7f4620003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f463966d640 1 -- 172.21.15.92:0/4131248438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46200051c0 msgr2=0x7f462000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f463966d640 1 --2- 172.21.15.92:0/4131248438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46200051c0 0x7f462000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f463966d640 1 -- 172.21.15.92:0/4131248438 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4634157440 con 0x7f4620004800 2024-04-22T07:04:10.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f463a66f640 1 -- 172.21.15.92:0/4131248438 reap_dead start 2024-04-22T07:04:10.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f7e311ab640 1 -- 172.21.15.92:0/2744678288 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0a9990 con 0x7f7e2404a790 2024-04-22T07:04:10.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f7e311ab640 1 -- 172.21.15.92:0/2744678288 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2c0a9b70 con 0x7f7e2404a790 2024-04-22T07:04:10.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f7e311ab640 1 -- 172.21.15.92:0/2744678288 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0b1420 con 0x7f7e2404a790 2024-04-22T07:04:10.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f3eeeffd640 1 -- 172.21.15.92:0/3603465456 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4034070 con 0x7f3ef006dee0 2024-04-22T07:04:10.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f9746591640 1 -- 172.21.15.92:0/980957643 >> v2:172.21.15.145:3300/0 conn(0x7f9740152ab0 msgr2=0x7f97400c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f9746591640 1 --2- 172.21.15.92:0/980957643 >> v2:172.21.15.145:3300/0 conn(0x7f9740152ab0 0x7f97400c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f9746591640 1 -- 172.21.15.92:0/980957643 >> v1:172.21.15.145:6789/0 conn(0x7f97400c37d0 legacy=0x7f97400713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f9746591640 1 -- 172.21.15.92:0/980957643 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974015c6e0 con 0x7f97400c4f50 2024-04-22T07:04:10.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f1a538b2640 1 Processor -- start 2024-04-22T07:04:10.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f3eeeffd640 1 -- 172.21.15.92:0/3603465456 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee402ed10 con 0x7f3ef006dee0 2024-04-22T07:04:10.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7efcd1fce640 1 --2- 172.21.15.92:0/1199294852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40dfa90 0x7efcc40071a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efcc4012080 tx=0x7efcbc02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b911aa6d895d943 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7efcc3fff640 1 -- 172.21.15.92:0/1199294852 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc034070 con 0x7efcc40dfa90 2024-04-22T07:04:10.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7efcc3fff640 1 -- 172.21.15.92:0/1199294852 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcbc030850 con 0x7efcc40dfa90 2024-04-22T07:04:10.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7efcc3fff640 1 -- 172.21.15.92:0/1199294852 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc030c20 con 0x7efcc40dfa90 2024-04-22T07:04:10.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.584+0000 7fc78e4c1640 1 Processor -- start 2024-04-22T07:04:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7fc78e4c1640 1 -- start start 2024-04-22T07:04:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7fc78e4c1640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc788159010 0x7fc78806e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7fc78e4c1640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 0x7fc78806d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7fc78e4c1640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc78815fa00 con 0x7fc788073b30 2024-04-22T07:04:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7fc78e4c1640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc788074070 con 0x7fc788159010 2024-04-22T07:04:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7fc78e4c1640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc7880c6c10 con 0x7fc7880c1a80 2024-04-22T07:04:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7fc7877fe640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fc788073b30 0x7fc7880c1370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:55208/0 (socket says 172.21.15.92:55208) 2024-04-22T07:04:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7fc7877fe640 1 -- 172.21.15.92:0/2605876215 learned_addr learned my addr 172.21.15.92:0/2605876215 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7fc78ca5e640 1 --2- 172.21.15.92:0/2605876215 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 0x7fc78806d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7ff7ada5c640 1 -- 172.21.15.92:0/893299968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7940039b0 msgr2=0x7ff794003dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:04:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7ff7ada5c640 1 --2- 172.21.15.92:0/893299968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7940039b0 0x7ff794003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7ff7ada5c640 1 -- 172.21.15.92:0/893299968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7940051c0 msgr2=0x7ff79400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7ff7ada5c640 1 --2- 172.21.15.92:0/893299968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7940051c0 0x7ff79400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7f3eeeffd640 1 -- 172.21.15.92:0/3603465456 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4038620 con 0x7f3ef006dee0 2024-04-22T07:04:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7f3ef6723640 1 -- 172.21.15.92:0/3603465456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef006dee0 msgr2=0x7f3ef00b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7f3ef6723640 1 --2- 172.21.15.92:0/3603465456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef006dee0 0x7f3ef00b1390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00700a0 tx=0x7f3ee402df60 comp rx=0 tx=0).stop 2024-04-22T07:04:10.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7ff7ada5c640 1 -- 172.21.15.92:0/893299968 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff794003520 con 0x7ff794004800 2024-04-22T07:04:10.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7ff7aea5e640 1 -- 172.21.15.92:0/893299968 reap_dead start 2024-04-22T07:04:10.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7ff7ada5c640 1 --2- 172.21.15.92:0/893299968 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff794004800 0x7ff794004c80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4001750 tx=0x7ff794005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13fb5f1868535cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7f7e34411640 1 -- 172.21.15.92:0/2744678288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2404a790 msgr2=0x7f7e2404ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.585+0000 7f1a538b2640 1 -- start start 2024-04-22T07:04:10.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7fc787fff640 1 --2- 172.21.15.92:0/2605876215 >> v2:172.21.15.145:3300/0 conn(0x7fc788159010 0x7fc78806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7f7e34411640 1 --2- 172.21.15.92:0/2744678288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2404a790 0x7f7e2404ac20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c02db50 tx=0x7f7e2c0a7b10 comp rx=0 tx=0).stop 2024-04-22T07:04:10.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7f3ef6723640 1 -- 172.21.15.92:0/3603465456 shutdown_connections 2024-04-22T07:04:10.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7f3ef6723640 1 --2- 172.21.15.92:0/3603465456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef006dee0 0x7f3ef00b1390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7f3ef6723640 1 --2- 172.21.15.92:0/3603465456 >> v2:172.21.15.145:3300/0 conn(0x7f3ef00c7c40 0x7f3ef00b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7f463966d640 1 --2- 172.21.15.92:0/4131248438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4620004800 0x7f4620004c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4624009ac0 tx=0x7f4620005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efd12e0d5d0c0088 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7f3ef6723640 1 -- 172.21.15.92:0/3603465456 >> 172.21.15.92:0/3603465456 conn(0x7f3ef006bfd0 msgr2=0x7f3ef006f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.586+0000 7f9746591640 1 --2- 172.21.15.92:0/980957643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97400c4f50 0x7f9740158bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f97400c5640 tx=0x7f973002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a391b172d5ed9791 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4001370 con 0x7ff794004800 2024-04-22T07:04:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7fc78ca5e640 1 -- 172.21.15.92:0/2605876215 >> v2:172.21.15.145:3300/0 conn(0x7fc788159010 msgr2=0x7fc78806e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7fc78ca5e640 1 --2- 172.21.15.92:0/2605876215 >> v2:172.21.15.145:3300/0 conn(0x7fc788159010 0x7fc78806e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7fc786ffd640 1 -- 172.21.15.92:0/2605876215 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 273394346 0 0) 0x7fc78815fa00 con 0x7fc788073b30 2024-04-22T07:04:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7fc78ca5e640 1 -- 172.21.15.92:0/2605876215 >> v1:172.21.15.145:6789/0 conn(0x7fc788073b30 legacy=0x7fc7880c1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f3ef6723640 1 -- 172.21.15.92:0/3603465456 shutdown_connections 2024-04-22T07:04:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7fc78ca5e640 1 -- 172.21.15.92:0/2605876215 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78815fca0 con 0x7fc7880c1a80 2024-04-22T07:04:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f1a538b2640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4c159080 0x7f1a4c162e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f1a538b2640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a4c15fca0 con 0x7f1a4c06e4e0 2024-04-22T07:04:10.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f1a538b2640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a4c06d340 con 0x7f1a4c06e0c0 2024-04-22T07:04:10.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f1a538b2640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c163360 con 0x7f1a4c159080 2024-04-22T07:04:10.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a40052f0 con 0x7ff794004800 2024-04-22T07:04:10.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7ff7ad25b640 1 -- 172.21.15.92:0/893299968 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4003830 con 0x7ff794004800 2024-04-22T07:04:10.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f3ef6723640 1 -- 172.21.15.92:0/3603465456 wait complete. 2024-04-22T07:04:10.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624039070 con 0x7f4620004800 2024-04-22T07:04:10.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f7e34411640 1 -- 172.21.15.92:0/2744678288 shutdown_connections 2024-04-22T07:04:10.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f7e34411640 1 --2- 172.21.15.92:0/2744678288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2404a790 0x7f7e2404ac20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f7e34411640 1 --2- 172.21.15.92:0/2744678288 >> v2:172.21.15.92:3300/0 conn(0x7f7e24050aa0 0x7f7e240e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f7e34411640 1 -- 172.21.15.92:0/2744678288 >> 172.21.15.92:0/2744678288 conn(0x7f7e24050fe0 msgr2=0x7f7e240513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f462402cc60 con 0x7f4620004800 2024-04-22T07:04:10.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f1a5164e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4c159080 0x7f1a4c162e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f1a51e4f640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1a4c06e4e0 0x7f1a4c15f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46680/0 (socket says 172.21.15.92:46680) 2024-04-22T07:04:10.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f1a51e4f640 1 -- 172.21.15.92:0/3198559867 learned_addr learned my addr 172.21.15.92:0/3198559867 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:10.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f4638e6c640 1 -- 172.21.15.92:0/4131248438 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624009690 con 0x7f4620004800 2024-04-22T07:04:10.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f9744d8e640 1 -- 172.21.15.92:0/980957643 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730035020 con 0x7f97400c4f50 2024-04-22T07:04:10.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f9744d8e640 1 -- 172.21.15.92:0/980957643 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9730031830 con 0x7f97400c4f50 2024-04-22T07:04:10.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f9744d8e640 1 -- 172.21.15.92:0/980957643 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730035020 con 0x7f97400c4f50 2024-04-22T07:04:10.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f7e34411640 1 -- 172.21.15.92:0/2744678288 shutdown_connections 2024-04-22T07:04:10.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7efcd27cf640 1 -- 172.21.15.92:0/1199294852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40dfa90 msgr2=0x7efcc40071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7efcd27cf640 1 --2- 172.21.15.92:0/1199294852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40dfa90 0x7efcc40071a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efcc4012080 tx=0x7efcbc02b990 comp rx=0 tx=0).stop 2024-04-22T07:04:10.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7efcd27cf640 1 -- 172.21.15.92:0/1199294852 shutdown_connections 2024-04-22T07:04:10.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7efcd27cf640 1 --2- 172.21.15.92:0/1199294852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40dfa90 0x7efcc40071a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7efcd27cf640 1 --2- 172.21.15.92:0/1199294852 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40e0480 0x7efcc404ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7efcd27cf640 1 --2- 172.21.15.92:0/1199294852 >> v2:172.21.15.145:3300/0 conn(0x7efcc40d9240 0x7efcc4039e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7efcd27cf640 1 -- 172.21.15.92:0/1199294852 >> 172.21.15.92:0/1199294852 conn(0x7efcc400e150 msgr2=0x7efcc400e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7efcd27cf640 1 -- 172.21.15.92:0/1199294852 shutdown_connections 2024-04-22T07:04:10.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7efcd27cf640 1 -- 172.21.15.92:0/1199294852 wait complete. 2024-04-22T07:04:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f463c0d2640 1 -- 172.21.15.92:0/4131248438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4620004800 msgr2=0x7f4620004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f463c0d2640 1 --2- 172.21.15.92:0/4131248438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4620004800 0x7f4620004c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4624009ac0 tx=0x7f4620005830 comp rx=0 tx=0).stop 2024-04-22T07:04:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f463c0d2640 1 -- 172.21.15.92:0/4131248438 shutdown_connections 2024-04-22T07:04:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f463c0d2640 1 --2- 172.21.15.92:0/4131248438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4620004800 0x7f4620004c80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f463c0d2640 1 -- 172.21.15.92:0/4131248438 >> 172.21.15.92:0/4131248438 conn(0x7f463406bfd0 msgr2=0x7f463406f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f9746d92640 1 -- 172.21.15.92:0/980957643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97400c4f50 msgr2=0x7f9740158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f9746d92640 1 --2- 172.21.15.92:0/980957643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97400c4f50 0x7f9740158bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f97400c5640 tx=0x7f973002b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f463c0d2640 1 -- 172.21.15.92:0/4131248438 shutdown_connections 2024-04-22T07:04:10.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f463c0d2640 1 -- 172.21.15.92:0/4131248438 wait complete. 2024-04-22T07:04:10.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f3ef6723640 1 Processor -- start 2024-04-22T07:04:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f1a5164e640 1 -- 172.21.15.92:0/3198559867 >> v1:172.21.15.145:6789/0 conn(0x7f1a4c06e0c0 legacy=0x7f1a4c15ee80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:04:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f1a5164e640 1 -- 172.21.15.92:0/3198559867 >> v1:172.21.15.92:6789/0 conn(0x7f1a4c06e4e0 legacy=0x7f1a4c15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f1a5164e640 1 -- 172.21.15.92:0/3198559867 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c0c6d20 con 0x7f1a4c159080 2024-04-22T07:04:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f9746d92640 1 -- 172.21.15.92:0/980957643 shutdown_connections 2024-04-22T07:04:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f9746d92640 1 --2- 172.21.15.92:0/980957643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97400c4f50 0x7f9740158bc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f9746d92640 1 --2- 172.21.15.92:0/980957643 >> v2:172.21.15.145:3300/0 conn(0x7f9740152ab0 0x7f97400c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7f9746d92640 1 -- 172.21.15.92:0/980957643 >> 172.21.15.92:0/980957643 conn(0x7f9740073830 msgr2=0x7f974006f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.587+0000 7f7e34411640 1 -- 172.21.15.92:0/2744678288 wait complete. 2024-04-22T07:04:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.588+0000 7fc78ca5e640 1 --2- 172.21.15.92:0/2605876215 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 0x7fc78806d340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc780009d90 tx=0x7fc78002bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e4981ea3ebca922 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7f1a5164e640 1 --2- 172.21.15.92:0/3198559867 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4c159080 0x7f1a4c162e20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009dc0 tx=0x7f1a4402bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab93bc4ab2114a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7f7e34411640 1 Processor -- start 2024-04-22T07:04:10.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7f1a43fff640 1 -- 172.21.15.92:0/3198559867 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44039070 con 0x7f1a4c159080 2024-04-22T07:04:10.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7ff7b04c1640 1 -- 172.21.15.92:0/893299968 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff794004800 msgr2=0x7ff794004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7ff7b04c1640 1 --2- 172.21.15.92:0/893299968 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff794004800 0x7ff794004c80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4001750 tx=0x7ff794005830 comp rx=0 tx=0).stop 2024-04-22T07:04:10.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7f9746d92640 1 -- 172.21.15.92:0/980957643 shutdown_connections 2024-04-22T07:04:10.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7f7e34411640 1 -- start start 2024-04-22T07:04:10.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7f1a43fff640 1 -- 172.21.15.92:0/3198559867 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a4402ca80 con 0x7f1a4c159080 2024-04-22T07:04:10.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7efcd27cf640 1 Processor -- start 2024-04-22T07:04:10.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7f1a43fff640 1 -- 172.21.15.92:0/3198559867 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4402ce50 con 0x7f1a4c159080 2024-04-22T07:04:10.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7efcd27cf640 1 -- start start 2024-04-22T07:04:10.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7f9746d92640 1 -- 172.21.15.92:0/980957643 wait complete. 2024-04-22T07:04:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7f7e34411640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e2404a790 0x7f7e2416d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f7e34411640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e24050aa0 0x7f7e2404df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f7e34411640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e240dd6d0 0x7f7e240cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f7e34411640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24050670 con 0x7f7e240dd6d0 2024-04-22T07:04:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f7e34411640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24049c80 con 0x7f7e2404a790 2024-04-22T07:04:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f7e34411640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7e240507f0 con 0x7f7e24050aa0 2024-04-22T07:04:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7fc786ffd640 1 -- 172.21.15.92:0/2605876215 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780039070 con 0x7fc7880c1a80 2024-04-22T07:04:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7fc786ffd640 1 -- 172.21.15.92:0/2605876215 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78002cc70 con 0x7fc7880c1a80 2024-04-22T07:04:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7fc786ffd640 1 -- 172.21.15.92:0/2605876215 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78002ce60 con 0x7fc7880c1a80 2024-04-22T07:04:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f1a538b2640 1 -- 172.21.15.92:0/3198559867 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4c159080 msgr2=0x7f1a4c162e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f1a538b2640 1 --2- 172.21.15.92:0/3198559867 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4c159080 0x7f1a4c162e20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009dc0 tx=0x7f1a4402bd50 comp rx=0 tx=0).stop 2024-04-22T07:04:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f7e319ac640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e24050aa0 0x7f7e2404df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f7e321ad640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e2404a790 0x7f7e2416d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f7e319ac640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e24050aa0 0x7f7e2404df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52398/0 (socket says 172.21.15.92:52398) 2024-04-22T07:04:10.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f7e319ac640 1 -- 172.21.15.92:0/77882887 learned_addr learned my addr 172.21.15.92:0/77882887 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7efcd27cf640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40d9240 0x7efcc4168380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.589+0000 7f3ef6723640 1 -- start start 2024-04-22T07:04:10.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f3ef6723640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ef006dee0 0x7f3ef00c3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f3ef6723640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef00c7c40 0x7f3ef00c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7efcd27cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40dfa90 0x7efcc416bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7efcd27cf640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcc40e0480 0x7efcc40481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7efcd27cf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efcc404faa0 con 0x7efcc40dfa90 2024-04-22T07:04:10.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7efcd27cf640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efcc404f7c0 con 0x7efcc40e0480 2024-04-22T07:04:10.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7efcd27cf640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efcc40dff20 con 0x7efcc40d9240 2024-04-22T07:04:10.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f3ef6723640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef00c8020 0x7f3ef0201360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7ff7b04c1640 1 -- 172.21.15.92:0/893299968 shutdown_connections 2024-04-22T07:04:10.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7ff7b04c1640 1 --2- 172.21.15.92:0/893299968 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff794004800 0x7ff794004c80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7ff7b04c1640 1 -- 172.21.15.92:0/893299968 >> 172.21.15.92:0/893299968 conn(0x7ff7a0051300 msgr2=0x7ff7a0051710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f7e329ae640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e240dd6d0 0x7f7e240cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7efcd0fcc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40dfa90 0x7efcc416bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7efcd17cd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40d9240 0x7efcc4168380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f3ef6723640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef00aef20 con 0x7f3ef00c8020 2024-04-22T07:04:10.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f3eef7fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef00c7c40 0x7f3ef00c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f3eef7fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef00c7c40 0x7f3ef00c3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:53812/0 (socket says 172.21.15.92:53812) 2024-04-22T07:04:10.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f3eef7fe640 1 -- 172.21.15.92:0/3886660171 learned_addr learned my addr 172.21.15.92:0/3886660171 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7efcd17cd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40d9240 0x7efcc4168380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52410/0 (socket says 172.21.15.92:52410) 2024-04-22T07:04:10.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7efcd17cd640 1 -- 172.21.15.92:0/1113841272 learned_addr learned my addr 172.21.15.92:0/1113841272 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f3ef6723640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0157a00 con 0x7f3ef00c7c40 2024-04-22T07:04:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f3ef6723640 1 -- 172.21.15.92:0/3886660171 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef00af0a0 con 0x7f3ef006dee0 2024-04-22T07:04:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f3ef4cc0640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef00c8020 0x7f3ef0201360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7efcd1fce640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcc40e0480 0x7efcc40481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.590+0000 7f463c0d2640 1 Processor -- start 2024-04-22T07:04:10.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7efcd17cd640 1 -- 172.21.15.92:0/1113841272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcc40e0480 msgr2=0x7efcc40481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f463c0d2640 1 -- start start 2024-04-22T07:04:10.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f3eeffff640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ef006dee0 0x7f3ef00c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f9746d92640 1 Processor -- start 2024-04-22T07:04:10.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7fc78e4c1640 1 -- 172.21.15.92:0/2605876215 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 msgr2=0x7fc78806d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f1a538b2640 1 -- 172.21.15.92:0/3198559867 shutdown_connections 2024-04-22T07:04:10.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f1a538b2640 1 --2- 172.21.15.92:0/3198559867 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4c159080 0x7f1a4c162e20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f1a538b2640 1 -- 172.21.15.92:0/3198559867 >> 172.21.15.92:0/3198559867 conn(0x7f1a4c06bfd0 msgr2=0x7f1a4c06f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7ff7b04c1640 1 -- 172.21.15.92:0/893299968 shutdown_connections 2024-04-22T07:04:10.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7fc78e4c1640 1 --2- 172.21.15.92:0/2605876215 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 0x7fc78806d340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc780009d90 tx=0x7fc78002bec0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f7e321ad640 1 -- 172.21.15.92:0/77882887 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e24050aa0 msgr2=0x7f7e2404df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f7e321ad640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e24050aa0 0x7f7e2404df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f7e321ad640 1 -- 172.21.15.92:0/77882887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e240dd6d0 msgr2=0x7f7e240cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7efcd17cd640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcc40e0480 0x7efcc40481f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7efcd17cd640 1 -- 172.21.15.92:0/1113841272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40dfa90 msgr2=0x7efcc416bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7efcd17cd640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40dfa90 0x7efcc416bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7efcd17cd640 1 -- 172.21.15.92:0/1113841272 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc41749d0 con 0x7efcc40d9240 2024-04-22T07:04:10.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7efcd1fce640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcc40e0480 0x7efcc40481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:10.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f3eef7fe640 1 -- 172.21.15.92:0/3886660171 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ef006dee0 msgr2=0x7f3ef00c3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f7e321ad640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e240dd6d0 0x7f7e240cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f7e321ad640 1 -- 172.21.15.92:0/77882887 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2416e0c0 con 0x7f7e2404a790 2024-04-22T07:04:10.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f3eef7fe640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ef006dee0 0x7f3ef00c3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f3eef7fe640 1 -- 172.21.15.92:0/3886660171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef00c8020 msgr2=0x7f3ef0201360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f3eef7fe640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef00c8020 0x7f3ef0201360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f3eef7fe640 1 -- 172.21.15.92:0/3886660171 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef00c5b00 con 0x7f3ef00c7c40 2024-04-22T07:04:10.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f3eeffff640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ef006dee0 0x7f3ef00c3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:10.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7fc78e4c1640 1 -- 172.21.15.92:0/2605876215 shutdown_connections 2024-04-22T07:04:10.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7fc78e4c1640 1 --2- 172.21.15.92:0/2605876215 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 0x7fc78806d340 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7fc78e4c1640 1 --2- 172.21.15.92:0/2605876215 >> v2:172.21.15.145:3300/0 conn(0x7fc788159010 0x7fc78806e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7fc78e4c1640 1 -- 172.21.15.92:0/2605876215 >> 172.21.15.92:0/2605876215 conn(0x7fc78806bfd0 msgr2=0x7fc78806f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7fc78e4c1640 1 -- 172.21.15.92:0/2605876215 shutdown_connections 2024-04-22T07:04:10.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7fc78e4c1640 1 -- 172.21.15.92:0/2605876215 wait complete. 2024-04-22T07:04:10.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f9746d92640 1 -- start start 2024-04-22T07:04:10.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f7e321ad640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e2404a790 0x7f7e2416d810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b410 tx=0x7f7e1c00b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f3eef7fe640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef00c7c40 0x7f3ef00c3cc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000c640 tx=0x7f3ee000ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7ff7b04c1640 1 -- 172.21.15.92:0/893299968 wait complete. 2024-04-22T07:04:10.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f1a538b2640 1 -- 172.21.15.92:0/3198559867 shutdown_connections 2024-04-22T07:04:10.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f1a538b2640 1 -- 172.21.15.92:0/3198559867 wait complete. 2024-04-22T07:04:10.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7efcd17cd640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40d9240 0x7efcc4168380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efccc06b060 tx=0x7efccc0695e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f7e22ffd640 1 -- 172.21.15.92:0/77882887 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c015020 con 0x7f7e2404a790 2024-04-22T07:04:10.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.591+0000 7f7e22ffd640 1 -- 172.21.15.92:0/77882887 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e1c011830 con 0x7f7e2404a790 2024-04-22T07:04:10.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f7e22ffd640 1 -- 172.21.15.92:0/77882887 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c011c20 con 0x7f7e2404a790 2024-04-22T07:04:10.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f7e34411640 1 -- 172.21.15.92:0/77882887 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2416e3a0 con 0x7f7e2404a790 2024-04-22T07:04:10.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f7e34411640 1 -- 172.21.15.92:0/77882887 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2416e960 con 0x7f7e2404a790 2024-04-22T07:04:10.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f9746d92640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97400c37d0 0x7f97400cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f9746d92640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97400c4f50 0x7f97400cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f9746d92640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9740152ab0 0x7f97400ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f463c0d2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4620004800 0x7f46340c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f9746d92640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9740152e90 con 0x7f97400c4f50 2024-04-22T07:04:10.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f463c0d2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4634150190 0x7f46340c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f9746d92640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f97400cbb80 con 0x7f9740152ab0 2024-04-22T07:04:10.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f9746d92640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9740153010 con 0x7f97400c37d0 2024-04-22T07:04:10.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f974558f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97400c4f50 0x7f97400cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f463c0d2640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634153180 0x7f4634153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f974558f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97400c4f50 0x7f97400cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36234/0 (socket says 172.21.15.92:36234) 2024-04-22T07:04:10.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f463c0d2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f46340aef20 con 0x7f4634150190 2024-04-22T07:04:10.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f974558f640 1 -- 172.21.15.92:0/735818435 learned_addr learned my addr 172.21.15.92:0/735818435 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f9746591640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9740152ab0 0x7f97400ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f463966d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4634150190 0x7f46340c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f463966d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4634150190 0x7f46340c5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36236/0 (socket says 172.21.15.92:36236) 2024-04-22T07:04:10.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f463966d640 1 -- 172.21.15.92:0/1039904449 learned_addr learned my addr 172.21.15.92:0/1039904449 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f463c0d2640 1 -- 172.21.15.92:0/1039904449 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4634157a00 con 0x7f4620004800 2024-04-22T07:04:10.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f463c0d2640 1 -- 172.21.15.92:0/1039904449 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f46340af0a0 con 0x7f4634153180 2024-04-22T07:04:10.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f4639e6e640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4620004800 0x7f46340c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f463a66f640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634153180 0x7f4634153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7ff7b04c1640 1 Processor -- start 2024-04-22T07:04:10.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7efcc1ffb640 1 -- 172.21.15.92:0/1113841272 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc095470 con 0x7efcc40d9240 2024-04-22T07:04:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7efcc1ffb640 1 -- 172.21.15.92:0/1113841272 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efccc090070 con 0x7efcc40d9240 2024-04-22T07:04:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7efcc1ffb640 1 -- 172.21.15.92:0/1113841272 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc095850 con 0x7efcc40d9240 2024-04-22T07:04:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f1a538b2640 1 Processor -- start 2024-04-22T07:04:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f3eecff9640 1 -- 172.21.15.92:0/3886660171 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0015020 con 0x7f3ef00c7c40 2024-04-22T07:04:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f3eecff9640 1 -- 172.21.15.92:0/3886660171 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee0011870 con 0x7f3ef00c7c40 2024-04-22T07:04:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.592+0000 7f3eecff9640 1 -- 172.21.15.92:0/3886660171 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0011c60 con 0x7f3ef00c7c40 2024-04-22T07:04:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f3ef6723640 1 -- 172.21.15.92:0/3886660171 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef0165710 con 0x7f3ef00c7c40 2024-04-22T07:04:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f463966d640 1 -- 172.21.15.92:0/1039904449 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634153180 msgr2=0x7f4634153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f463966d640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634153180 0x7f4634153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f463966d640 1 -- 172.21.15.92:0/1039904449 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4620004800 msgr2=0x7f46340c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f463966d640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4620004800 0x7f46340c35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f463966d640 1 -- 172.21.15.92:0/1039904449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4634153b40 con 0x7f4634150190 2024-04-22T07:04:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f463a66f640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634153180 0x7f4634153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f3ef6723640 1 -- 172.21.15.92:0/3886660171 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef00c6060 con 0x7f3ef00c7c40 2024-04-22T07:04:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f1a538b2640 1 -- start start 2024-04-22T07:04:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7efcd27cf640 1 -- 172.21.15.92:0/1113841272 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc416b170 con 0x7efcc40d9240 2024-04-22T07:04:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7efcd27cf640 1 -- 172.21.15.92:0/1113841272 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc4174c50 con 0x7efcc40d9240 2024-04-22T07:04:10.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7fc78e4c1640 1 Processor -- start 2024-04-22T07:04:10.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7fc78e4c1640 1 -- start start 2024-04-22T07:04:10.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7f9745d90640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97400c37d0 0x7f97400cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f7e34411640 1 -- 172.21.15.92:0/77882887 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e24005380 con 0x7f7e2404a790 2024-04-22T07:04:10.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.593+0000 7ff7b04c1640 1 -- start start 2024-04-22T07:04:10.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7ff7b04c1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff794004800 0x7ff7a00d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7ff7b04c1640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7a00db3d0 0x7ff7a00d8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7ff7b04c1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a00e1e10 0x7ff7a00d63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7ff7b04c1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a0047800 con 0x7ff7a00e1e10 2024-04-22T07:04:10.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7ff7b04c1640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a0047520 con 0x7ff794004800 2024-04-22T07:04:10.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7ff7b04c1640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a0051e20 con 0x7ff7a00db3d0 2024-04-22T07:04:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f1a538b2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a4c06e0c0 0x7f1a4c072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f9746591640 1 -- 172.21.15.92:0/735818435 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97400c37d0 msgr2=0x7f97400cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f9746591640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97400c37d0 0x7f97400cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f9746591640 1 -- 172.21.15.92:0/735818435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97400c4f50 msgr2=0x7f97400cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f9746591640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97400c4f50 0x7f97400cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f9746591640 1 -- 172.21.15.92:0/735818435 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97401ed060 con 0x7f9740152ab0 2024-04-22T07:04:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f974558f640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97400c4f50 0x7f97400cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f1a538b2640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4c06e4e0 0x7f1a4c1f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7ff7ae25d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff794004800 0x7ff7a00d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f1a538b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a4c159080 0x7f1a4c1f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f1a538b2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c0c72c0 con 0x7f1a4c159080 2024-04-22T07:04:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f1a538b2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c0c6fe0 con 0x7f1a4c06e0c0 2024-04-22T07:04:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f1a538b2640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c06d5e0 con 0x7f1a4c06e4e0 2024-04-22T07:04:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7ff7ae25d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff794004800 0x7ff7a00d29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:53856/0 (socket says 172.21.15.92:53856) 2024-04-22T07:04:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7ff7ae25d640 1 -- 172.21.15.92:0/1200396898 learned_addr learned my addr 172.21.15.92:0/1200396898 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7efcd27cf640 1 -- 172.21.15.92:0/1113841272 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc40057d0 con 0x7efcc40d9240 2024-04-22T07:04:10.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f1a5164e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a4c06e0c0 0x7f1a4c072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7fc78e4c1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc788073b30 0x7fc7880c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7fc78e4c1640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 0x7fc7880b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f1a5164e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a4c06e0c0 0x7f1a4c072bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:53860/0 (socket says 172.21.15.92:53860) 2024-04-22T07:04:10.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7f1a5164e640 1 -- 172.21.15.92:0/3815836870 learned_addr learned my addr 172.21.15.92:0/3815836870 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f1a51e4f640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a4c159080 0x7f1a4c1f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.594+0000 7fc78e4c1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc788159010 0x7fc7880b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f1a50e4d640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4c06e4e0 0x7f1a4c1f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7ff7ada5c640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7a00db3d0 0x7ff7a00d8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7ff7aea5e640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a00e1e10 0x7ff7a00d63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f463966d640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4634150190 0x7f46340c5e80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4624009ac0 tx=0x7f4624009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f1a50e4d640 1 -- 172.21.15.92:0/3815836870 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a4c06e0c0 msgr2=0x7f1a4c072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f1a50e4d640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a4c06e0c0 0x7f1a4c072bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f1a50e4d640 1 -- 172.21.15.92:0/3815836870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a4c159080 msgr2=0x7f1a4c1f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc7877fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 0x7fc7880b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc7877fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 0x7fc7880b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52480/0 (socket says 172.21.15.92:52480) 2024-04-22T07:04:10.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc7877fe640 1 -- 172.21.15.92:0/1554327478 learned_addr learned my addr 172.21.15.92:0/1554327478 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:10.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881593f0 con 0x7fc788159010 2024-04-22T07:04:10.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc7880c71b0 con 0x7fc788073b30 2024-04-22T07:04:10.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc788159570 con 0x7fc7880c1a80 2024-04-22T07:04:10.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc787fff640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc788073b30 0x7fc7880c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc78ca5e640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc788159010 0x7fc7880b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f1a50e4d640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a4c159080 0x7f1a4c1f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f1a50e4d640 1 -- 172.21.15.92:0/3815836870 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c155950 con 0x7f1a4c06e4e0 2024-04-22T07:04:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f7e22ffd640 1 -- 172.21.15.92:0/77882887 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e1c018020 con 0x7f7e2404a790 2024-04-22T07:04:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f7e22ffd640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7e1006c9c0 0x7f7e1006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc7877fe640 1 -- 172.21.15.92:0/1554327478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc788073b30 msgr2=0x7fc7880c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc7877fe640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc788073b30 0x7fc7880c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc7877fe640 1 -- 172.21.15.92:0/1554327478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc788159010 msgr2=0x7fc7880b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc7877fe640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc788159010 0x7fc7880b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc7877fe640 1 -- 172.21.15.92:0/1554327478 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7880c8d00 con 0x7fc7880c1a80 2024-04-22T07:04:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f3ef6723640 1 -- 172.21.15.92:0/3886660171 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef006e3e0 con 0x7f3ef00c7c40 2024-04-22T07:04:10.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7fc78ca5e640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc788159010 0x7fc7880b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:10.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7ff7ada5c640 1 -- 172.21.15.92:0/1200396898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff794004800 msgr2=0x7ff7a00d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f462a7fc640 1 -- 172.21.15.92:0/1039904449 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624039030 con 0x7f4634150190 2024-04-22T07:04:10.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7ff7ada5c640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff794004800 0x7ff7a00d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7ff7ada5c640 1 -- 172.21.15.92:0/1200396898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a00e1e10 msgr2=0x7ff7a00d63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7ff7ada5c640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a00e1e10 0x7ff7a00d63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7ff7ada5c640 1 -- 172.21.15.92:0/1200396898 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a00d2f30 con 0x7ff7a00db3d0 2024-04-22T07:04:10.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7f9746591640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9740152ab0 0x7f97400ccf80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9730033000 tx=0x7f97300046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7f462a7fc640 1 -- 172.21.15.92:0/1039904449 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4624043070 con 0x7f4634150190 2024-04-22T07:04:10.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7f463c0d2640 1 -- 172.21.15.92:0/1039904449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46341f0300 con 0x7f4634150190 2024-04-22T07:04:10.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7f462a7fc640 1 -- 172.21.15.92:0/1039904449 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624003ea0 con 0x7f4634150190 2024-04-22T07:04:10.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7ff7ada5c640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7a00db3d0 0x7ff7a00d8470 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff79400c3e0 tx=0x7ff7a4001120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7f463c0d2640 1 -- 172.21.15.92:0/1039904449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46341f08c0 con 0x7f4634150190 2024-04-22T07:04:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7efcc1ffb640 1 -- 172.21.15.92:0/1113841272 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efccc0959f0 con 0x7efcc40d9240 2024-04-22T07:04:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.595+0000 7f7e22ffd640 1 -- 172.21.15.92:0/77882887 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e1c08b890 con 0x7f7e2404a790 2024-04-22T07:04:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7f7e319ac640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7e1006c9c0 0x7f7e1006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7f3eecff9640 1 -- 172.21.15.92:0/3886660171 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ee0018020 con 0x7f3ef00c7c40 2024-04-22T07:04:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7f1a50e4d640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4c06e4e0 0x7f1a4c1f88b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800b410 tx=0x7f1a4800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7efcc1ffb640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efcb006c9c0 0x7efcb006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7efcc1ffb640 1 -- 172.21.15.92:0/1113841272 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efccc1078e0 con 0x7efcc40d9240 2024-04-22T07:04:10.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7efcd0fcc640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efcb006c9c0 0x7efcb006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7f7e319ac640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7e1006c9c0 0x7f7e1006ee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e2403a4b0 tx=0x7f7e28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7f3eecff9640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3edc06c940 0x7f3edc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.596+0000 7fc7877fe640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 0x7fc7880b0760 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c420 tx=0x7fc77c00c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.597+0000 7f3eecff9640 1 -- 172.21.15.92:0/3886660171 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3ee008bad0 con 0x7f3ef00c7c40 2024-04-22T07:04:10.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.597+0000 7f3eeffff640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3edc06c940 0x7f3edc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.597+0000 7efcd0fcc640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efcb006c9c0 0x7efcb006ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efcc4047980 tx=0x7efcc8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.597+0000 7f463c0d2640 1 -- 172.21.15.92:0/1039904449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461c005290 con 0x7f4634150190 2024-04-22T07:04:10.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.597+0000 7f3eeffff640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3edc06c940 0x7f3edc06ee00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009fc0 tx=0x7f3ee8009550 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.597+0000 7f462a7fc640 1 -- 172.21.15.92:0/1039904449 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f462403e660 con 0x7f4634150190 2024-04-22T07:04:10.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.598+0000 7f462a7fc640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f461806c970 0x7f461806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.598+0000 7f4639e6e640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f461806c970 0x7f461806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.598+0000 7f462a7fc640 1 -- 172.21.15.92:0/1039904449 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f46240add70 con 0x7f4634150190 2024-04-22T07:04:10.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.598+0000 7f4639e6e640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f461806c970 0x7f461806ee30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f46300079d0 tx=0x7f4630009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7f97367fc640 1 -- 172.21.15.92:0/735818435 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730035020 con 0x7f9740152ab0 2024-04-22T07:04:10.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7ff79effd640 1 -- 172.21.15.92:0/1200396898 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4041020 con 0x7ff7a00db3d0 2024-04-22T07:04:10.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7f1a41ffb640 1 -- 172.21.15.92:0/3815836870 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48015020 con 0x7f1a4c06e4e0 2024-04-22T07:04:10.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7f9746d92640 1 -- 172.21.15.92:0/735818435 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97400cd680 con 0x7f9740152ab0 2024-04-22T07:04:10.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7f1a538b2640 1 -- 172.21.15.92:0/3815836870 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4c154f20 con 0x7f1a4c06e4e0 2024-04-22T07:04:10.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7f9746d92640 1 -- 172.21.15.92:0/735818435 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974006e530 con 0x7f9740152ab0 2024-04-22T07:04:10.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7f1a41ffb640 1 -- 172.21.15.92:0/3815836870 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a48011850 con 0x7f1a4c06e4e0 2024-04-22T07:04:10.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7f1a41ffb640 1 -- 172.21.15.92:0/3815836870 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48011c40 con 0x7f1a4c06e4e0 2024-04-22T07:04:10.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7f1a538b2640 1 -- 172.21.15.92:0/3815836870 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4c155c10 con 0x7f1a4c06e4e0 2024-04-22T07:04:10.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7efcc1ffb640 1 -- 172.21.15.92:0/1113841272 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcc41749d0 con 0x7efcc40d9240 2024-04-22T07:04:10.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7fc784ff9640 1 -- 172.21.15.92:0/1554327478 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0074e0 con 0x7fc7880c1a80 2024-04-22T07:04:10.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.601+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7880c95c0 con 0x7fc7880c1a80 2024-04-22T07:04:10.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.601+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc788155830 con 0x7fc7880c1a80 2024-04-22T07:04:10.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7ff7b04c1640 1 -- 172.21.15.92:0/1200396898 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a0047d30 con 0x7ff7a00db3d0 2024-04-22T07:04:10.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.600+0000 7ff7b04c1640 1 -- 172.21.15.92:0/1200396898 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a0047fd0 con 0x7ff7a00db3d0 2024-04-22T07:04:10.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.601+0000 7ff79effd640 1 -- 172.21.15.92:0/1200396898 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a40054a0 con 0x7ff7a00db3d0 2024-04-22T07:04:10.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.601+0000 7ff79effd640 1 -- 172.21.15.92:0/1200396898 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4006c40 con 0x7ff7a00db3d0 2024-04-22T07:04:10.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.601+0000 7f7e22ffd640 1 -- 172.21.15.92:0/77882887 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e1c01f020 con 0x7f7e2404a790 2024-04-22T07:04:10.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.601+0000 7fc784ff9640 1 -- 172.21.15.92:0/1554327478 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77c0047e0 con 0x7fc7880c1a80 2024-04-22T07:04:10.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.601+0000 7fc784ff9640 1 -- 172.21.15.92:0/1554327478 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c004c10 con 0x7fc7880c1a80 2024-04-22T07:04:10.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.602+0000 7ff7b04c1640 1 -- 172.21.15.92:0/1200396898 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff790005290 con 0x7ff7a00db3d0 2024-04-22T07:04:10.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.602+0000 7f1a538b2640 1 -- 172.21.15.92:0/3815836870 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a4c155500 con 0x7f1a4c06e4e0 2024-04-22T07:04:10.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.602+0000 7f9746d92640 1 -- 172.21.15.92:0/735818435 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f974004fdc0 con 0x7f9740152ab0 2024-04-22T07:04:10.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.602+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc768005290 con 0x7fc7880c1a80 2024-04-22T07:04:10.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.602+0000 7f1a41ffb640 1 -- 172.21.15.92:0/3815836870 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a48018020 con 0x7f1a4c06e4e0 2024-04-22T07:04:10.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.603+0000 7ff79effd640 1 -- 172.21.15.92:0/1200396898 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7a4044020 con 0x7ff7a00db3d0 2024-04-22T07:04:10.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.603+0000 7f1a41ffb640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1a3006c920 0x7f1a3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.603+0000 7f1a41ffb640 1 -- 172.21.15.92:0/3815836870 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1a4808b7f0 con 0x7f1a4c06e4e0 2024-04-22T07:04:10.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.603+0000 7ff79effd640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff78c06c920 0x7ff78c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.603+0000 7f1a5164e640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1a3006c920 0x7f1a3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.604+0000 7ff79effd640 1 -- 172.21.15.92:0/1200396898 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff7a40ad720 con 0x7ff7a00db3d0 2024-04-22T07:04:10.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.604+0000 7ff7ae25d640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff78c06c920 0x7ff78c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.604+0000 7fc784ff9640 1 -- 172.21.15.92:0/1554327478 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc77c011070 con 0x7fc7880c1a80 2024-04-22T07:04:10.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.604+0000 7f462a7fc640 1 -- 172.21.15.92:0/1039904449 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4634153b40 con 0x7f4634150190 2024-04-22T07:04:10.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.604+0000 7ff7ae25d640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff78c06c920 0x7ff78c06ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff7a807bca0 tx=0x7ff7a806e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.604+0000 7f1a5164e640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1a3006c920 0x7f1a3006ede0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009bd0 tx=0x7f1a44030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.604+0000 7fc784ff9640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc77406c8f0 0x7fc77406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.604+0000 7fc784ff9640 1 -- 172.21.15.92:0/1554327478 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc77c08b920 con 0x7fc7880c1a80 2024-04-22T07:04:10.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.604+0000 7fc787fff640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc77406c8f0 0x7fc77406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.605+0000 7fc787fff640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc77406c8f0 0x7fc77406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc778009a10 tx=0x7fc778008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.605+0000 7f97367fc640 1 -- 172.21.15.92:0/735818435 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9730031830 con 0x7f9740152ab0 2024-04-22T07:04:10.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.605+0000 7f97367fc640 1 -- 172.21.15.92:0/735818435 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730053060 con 0x7f9740152ab0 2024-04-22T07:04:10.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.605+0000 7f3eecff9640 1 -- 172.21.15.92:0/3886660171 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ef006e3e0 con 0x7f3ef00c7c40 2024-04-22T07:04:10.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.607+0000 7f97367fc640 1 -- 172.21.15.92:0/735818435 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9730038020 con 0x7f9740152ab0 2024-04-22T07:04:10.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.607+0000 7f97367fc640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f972406c8f0 0x7f972406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.607+0000 7ff79effd640 1 -- 172.21.15.92:0/1200396898 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7a404b030 con 0x7ff7a00db3d0 2024-04-22T07:04:10.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.607+0000 7f97367fc640 1 -- 172.21.15.92:0/735818435 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f97300adf20 con 0x7f9740152ab0 2024-04-22T07:04:10.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.607+0000 7f9745d90640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f972406c8f0 0x7f972406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.608+0000 7f9745d90640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f972406c8f0 0x7f972406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f973c009970 tx=0x7f973c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.610+0000 7f97367fc640 1 -- 172.21.15.92:0/735818435 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9730040070 con 0x7f9740152ab0 2024-04-22T07:04:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.612+0000 7f1a41ffb640 1 -- 172.21.15.92:0/3815836870 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a4801f030 con 0x7f1a4c06e4e0 2024-04-22T07:04:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:10.616+0000 7fc784ff9640 1 -- 172.21.15.92:0/1554327478 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc77c059d30 con 0x7fc7880c1a80 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.023+0000 7efcd27cf640 1 -- 172.21.15.92:0/1113841272 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7efcc400eff0 con 0x7efcc40d9240 2024-04-22T07:04:11.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.023+0000 7efcc1ffb640 1 -- 172.21.15.92:0/1113841272 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efccc0d5610 con 0x7efcc40d9240 2024-04-22T07:04:11.021 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:11.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7efcaf7fe640 1 -- 172.21.15.92:0/1113841272 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efcb006c9c0 msgr2=0x7efcb006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7f7e34411640 1 -- 172.21.15.92:0/77882887 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7e2400de20 con 0x7f7e2404a790 2024-04-22T07:04:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7efcaf7fe640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efcb006c9c0 0x7efcb006ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efcc4047980 tx=0x7efcc8008040 comp rx=0 tx=0).stop 2024-04-22T07:04:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7efcaf7fe640 1 -- 172.21.15.92:0/1113841272 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40d9240 msgr2=0x7efcc4168380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7efcaf7fe640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40d9240 0x7efcc4168380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efccc06b060 tx=0x7efccc0695e0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7efcaf7fe640 1 -- 172.21.15.92:0/1113841272 shutdown_connections 2024-04-22T07:04:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7efcaf7fe640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7efcb006c9c0 0x7efcb006ee80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7efcaf7fe640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcc40e0480 0x7efcc40481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7efcaf7fe640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcc40dfa90 0x7efcc416bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7efcaf7fe640 1 --2- 172.21.15.92:0/1113841272 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efcc40d9240 0x7efcc4168380 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7efcaf7fe640 1 -- 172.21.15.92:0/1113841272 >> 172.21.15.92:0/1113841272 conn(0x7efcc400e150 msgr2=0x7efcc40d8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7efcaf7fe640 1 -- 172.21.15.92:0/1113841272 shutdown_connections 2024-04-22T07:04:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.027+0000 7efcaf7fe640 1 -- 172.21.15.92:0/1113841272 wait complete. 2024-04-22T07:04:11.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.028+0000 7f7e22ffd640 1 -- 172.21.15.92:0/77882887 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7e1c0596e0 con 0x7f7e2404a790 2024-04-22T07:04:11.027 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2024-04-22T07:04:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.031+0000 7f7e20ff9640 1 -- 172.21.15.92:0/77882887 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7e1006c9c0 msgr2=0x7f7e1006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.031+0000 7f7e20ff9640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7e1006c9c0 0x7f7e1006ee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e2403a4b0 tx=0x7f7e28008040 comp rx=0 tx=0).stop 2024-04-22T07:04:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.031+0000 7f7e20ff9640 1 -- 172.21.15.92:0/77882887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e2404a790 msgr2=0x7f7e2416d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.032+0000 7f7e20ff9640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e2404a790 0x7f7e2416d810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b410 tx=0x7f7e1c00b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.032+0000 7f7e20ff9640 1 -- 172.21.15.92:0/77882887 shutdown_connections 2024-04-22T07:04:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.032+0000 7f7e20ff9640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7e1006c9c0 0x7f7e1006ee80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.032+0000 7f7e20ff9640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e240dd6d0 0x7f7e240cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.032+0000 7f7e20ff9640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e24050aa0 0x7f7e2404df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.032+0000 7f7e20ff9640 1 --2- 172.21.15.92:0/77882887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e2404a790 0x7f7e2416d810 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.032+0000 7f7e20ff9640 1 -- 172.21.15.92:0/77882887 >> 172.21.15.92:0/77882887 conn(0x7f7e24050fe0 msgr2=0x7f7e24053ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.032+0000 7f7e20ff9640 1 -- 172.21.15.92:0/77882887 shutdown_connections 2024-04-22T07:04:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.032+0000 7f7e20ff9640 1 -- 172.21.15.92:0/77882887 wait complete. 2024-04-22T07:04:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.032+0000 7ff7b04c1640 1 -- 172.21.15.92:0/1200396898 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff790005850 con 0x7ff7a00db3d0 2024-04-22T07:04:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.033+0000 7ff79effd640 1 -- 172.21.15.92:0/1200396898 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff7a407bbc0 con 0x7ff7a00db3d0 2024-04-22T07:04:11.032 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:11.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.034+0000 7f3ef6723640 1 -- 172.21.15.92:0/3886660171 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3ef006c6f0 con 0x7f3ef00c7c40 2024-04-22T07:04:11.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.034+0000 7f3eecff9640 1 -- 172.21.15.92:0/3886660171 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ef006c6f0 con 0x7f3ef00c7c40 2024-04-22T07:04:11.034 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2024-04-22T07:04:11.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.036+0000 7f9746d92640 1 -- 172.21.15.92:0/735818435 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f97400cb7b0 con 0x7f9740152ab0 2024-04-22T07:04:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.036+0000 7ff7b04c1640 1 -- 172.21.15.92:0/1200396898 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff78c06c920 msgr2=0x7ff78c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.036+0000 7ff7b04c1640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff78c06c920 0x7ff78c06ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff7a807bca0 tx=0x7ff7a806e040 comp rx=0 tx=0).stop 2024-04-22T07:04:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.036+0000 7ff7b04c1640 1 -- 172.21.15.92:0/1200396898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7a00db3d0 msgr2=0x7ff7a00d8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.036+0000 7ff7b04c1640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7a00db3d0 0x7ff7a00d8470 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff79400c3e0 tx=0x7ff7a4001120 comp rx=0 tx=0).stop 2024-04-22T07:04:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.037+0000 7ff7b04c1640 1 -- 172.21.15.92:0/1200396898 shutdown_connections 2024-04-22T07:04:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.037+0000 7ff7b04c1640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff78c06c920 0x7ff78c06ede0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.037+0000 7ff7b04c1640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a00e1e10 0x7ff7a00d63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.037+0000 7ff7b04c1640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7a00db3d0 0x7ff7a00d8470 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.037+0000 7ff7b04c1640 1 --2- 172.21.15.92:0/1200396898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff794004800 0x7ff7a00d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.037+0000 7ff7b04c1640 1 -- 172.21.15.92:0/1200396898 >> 172.21.15.92:0/1200396898 conn(0x7ff7a0051300 msgr2=0x7ff7a0050790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.037+0000 7ff7b04c1640 1 -- 172.21.15.92:0/1200396898 shutdown_connections 2024-04-22T07:04:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.037+0000 7ff7b04c1640 1 -- 172.21.15.92:0/1200396898 wait complete. 2024-04-22T07:04:11.037 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:11.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.037+0000 7f97367fc640 1 -- 172.21.15.92:0/735818435 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f973007bd00 con 0x7f9740152ab0 2024-04-22T07:04:11.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.039+0000 7f3ef6723640 1 -- 172.21.15.92:0/3886660171 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3edc06c940 msgr2=0x7f3edc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.039+0000 7f3ef6723640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3edc06c940 0x7f3edc06ee00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009fc0 tx=0x7f3ee8009550 comp rx=0 tx=0).stop 2024-04-22T07:04:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.039+0000 7f3ef6723640 1 -- 172.21.15.92:0/3886660171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef00c7c40 msgr2=0x7f3ef00c3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.039+0000 7f3ef6723640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef00c7c40 0x7f3ef00c3cc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000c640 tx=0x7f3ee000ca40 comp rx=0 tx=0).stop 2024-04-22T07:04:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.039+0000 7f3ef6723640 1 -- 172.21.15.92:0/3886660171 shutdown_connections 2024-04-22T07:04:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.039+0000 7f3ef6723640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3edc06c940 0x7f3edc06ee00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.039+0000 7f3ef6723640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef00c8020 0x7f3ef0201360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.039+0000 7f3ef6723640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef00c7c40 0x7f3ef00c3cc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.039+0000 7f3ef6723640 1 --2- 172.21.15.92:0/3886660171 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ef006dee0 0x7f3ef00c3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.039+0000 7f3ef6723640 1 -- 172.21.15.92:0/3886660171 >> 172.21.15.92:0/3886660171 conn(0x7f3ef006bfd0 msgr2=0x7f3ef00b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.039+0000 7f3ef6723640 1 -- 172.21.15.92:0/3886660171 shutdown_connections 2024-04-22T07:04:11.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.039+0000 7f3ef6723640 1 -- 172.21.15.92:0/3886660171 wait complete. 2024-04-22T07:04:11.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.040+0000 7f9746d92640 1 -- 172.21.15.92:0/735818435 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f972406c8f0 msgr2=0x7f972406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.040+0000 7f9746d92640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f972406c8f0 0x7f972406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f973c009970 tx=0x7f973c008040 comp rx=0 tx=0).stop 2024-04-22T07:04:11.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.041+0000 7f9746d92640 1 -- 172.21.15.92:0/735818435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9740152ab0 msgr2=0x7f97400ccf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.041+0000 7f9746d92640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9740152ab0 0x7f97400ccf80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9730033000 tx=0x7f97300046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.041+0000 7f9746d92640 1 -- 172.21.15.92:0/735818435 shutdown_connections 2024-04-22T07:04:11.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.041+0000 7f9746d92640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f972406c8f0 0x7f972406edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.041+0000 7f9746d92640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9740152ab0 0x7f97400ccf80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.041+0000 7f9746d92640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97400c4f50 0x7f97400cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.041+0000 7f9746d92640 1 --2- 172.21.15.92:0/735818435 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97400c37d0 0x7f97400cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.041+0000 7f9746d92640 1 -- 172.21.15.92:0/735818435 >> 172.21.15.92:0/735818435 conn(0x7f9740073830 msgr2=0x7f974014fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.041+0000 7f9746d92640 1 -- 172.21.15.92:0/735818435 shutdown_connections 2024-04-22T07:04:11.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.041+0000 7f9746d92640 1 -- 172.21.15.92:0/735818435 wait complete. 2024-04-22T07:04:11.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.041+0000 7f463c0d2640 1 -- 172.21.15.92:0/1039904449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f461c005580 con 0x7f4634150190 2024-04-22T07:04:11.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.042+0000 7f462a7fc640 1 -- 172.21.15.92:0/1039904449 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f462407c210 con 0x7f4634150190 2024-04-22T07:04:11.041 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2024-04-22T07:04:11.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.043+0000 7f1a538b2640 1 -- 172.21.15.92:0/3815836870 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1a4c04fdc0 con 0x7f1a4c06e4e0 2024-04-22T07:04:11.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.043+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc768005850 con 0x7fc7880c1a80 2024-04-22T07:04:11.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.043+0000 7f1a41ffb640 1 -- 172.21.15.92:0/3815836870 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1a48059af0 con 0x7f1a4c06e4e0 2024-04-22T07:04:11.042 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2024-04-22T07:04:11.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.044+0000 7fc784ff9640 1 -- 172.21.15.92:0/1554327478 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc77c059630 con 0x7fc7880c1a80 2024-04-22T07:04:11.043 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2024-04-22T07:04:11.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-04-22T07:04:11.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f463c0d2640 1 -- 172.21.15.92:0/1039904449 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f461806c970 msgr2=0x7f461806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f463c0d2640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f461806c970 0x7f461806ee30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f46300079d0 tx=0x7f4630009040 comp rx=0 tx=0).stop 2024-04-22T07:04:11.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f463c0d2640 1 -- 172.21.15.92:0/1039904449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4634150190 msgr2=0x7f46340c5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f463c0d2640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4634150190 0x7f46340c5e80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4624009ac0 tx=0x7f4624009190 comp rx=0 tx=0).stop 2024-04-22T07:04:11.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f463c0d2640 1 -- 172.21.15.92:0/1039904449 shutdown_connections 2024-04-22T07:04:11.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f463c0d2640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f461806c970 0x7f461806ee30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f463c0d2640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4634153180 0x7f4634153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f1a538b2640 1 -- 172.21.15.92:0/3815836870 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1a3006c920 msgr2=0x7f1a3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f1a538b2640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1a3006c920 0x7f1a3006ede0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009bd0 tx=0x7f1a44030000 comp rx=0 tx=0).stop 2024-04-22T07:04:11.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f463c0d2640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4634150190 0x7f46340c5e80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc77406c8f0 msgr2=0x7fc77406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f1a538b2640 1 -- 172.21.15.92:0/3815836870 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4c06e4e0 msgr2=0x7f1a4c1f88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7fc78e4c1640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc77406c8f0 0x7fc77406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc778009a10 tx=0x7fc778008040 comp rx=0 tx=0).stop 2024-04-22T07:04:11.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f1a538b2640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4c06e4e0 0x7f1a4c1f88b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800b410 tx=0x7f1a4800b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f463c0d2640 1 --2- 172.21.15.92:0/1039904449 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4620004800 0x7f46340c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7f463c0d2640 1 -- 172.21.15.92:0/1039904449 >> 172.21.15.92:0/1039904449 conn(0x7f463406bfd0 msgr2=0x7f4634071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 msgr2=0x7fc7880b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7fc78e4c1640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 0x7fc7880b0760 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c420 tx=0x7fc77c00c8e0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 shutdown_connections 2024-04-22T07:04:11.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7fc78e4c1640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc77406c8f0 0x7fc77406edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.047+0000 7fc78e4c1640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc788159010 0x7fc7880b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7fc78e4c1640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7880c1a80 0x7fc7880b0760 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7fc78e4c1640 1 --2- 172.21.15.92:0/1554327478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc788073b30 0x7fc7880c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 >> 172.21.15.92:0/1554327478 conn(0x7fc78806bfd0 msgr2=0x7fc788156600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7f1a538b2640 1 -- 172.21.15.92:0/3815836870 shutdown_connections 2024-04-22T07:04:11.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7f1a538b2640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1a3006c920 0x7f1a3006ede0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7f1a538b2640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a4c159080 0x7f1a4c1f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7f1a538b2640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4c06e4e0 0x7f1a4c1f88b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7f1a538b2640 1 --2- 172.21.15.92:0/3815836870 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a4c06e0c0 0x7f1a4c072bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7f1a538b2640 1 -- 172.21.15.92:0/3815836870 >> 172.21.15.92:0/3815836870 conn(0x7f1a4c06bfd0 msgr2=0x7f1a4c071df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 shutdown_connections 2024-04-22T07:04:11.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7fc78e4c1640 1 -- 172.21.15.92:0/1554327478 wait complete. 2024-04-22T07:04:11.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7f463c0d2640 1 -- 172.21.15.92:0/1039904449 shutdown_connections 2024-04-22T07:04:11.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7f463c0d2640 1 -- 172.21.15.92:0/1039904449 wait complete. 2024-04-22T07:04:11.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7f1a538b2640 1 -- 172.21.15.92:0/3815836870 shutdown_connections 2024-04-22T07:04:11.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:11.048+0000 7f1a538b2640 1 -- 172.21.15.92:0/3815836870 wait complete. 2024-04-22T07:04:11.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-04-22T07:04:11.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-04-22T07:04:11.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-04-22T07:04:11.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-04-22T07:04:11.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2024-04-22T07:04:11.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-04-22T07:04:11.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2024-04-22T07:04:12.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-22T07:04:12.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-22T07:04:12.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-22T07:04:12.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-22T07:04:12.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-22T07:04:12.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-22T07:04:12.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-22T07:04:12.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-22T07:04:12.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.199+0000 7fda6ffff640 1 Processor -- start 2024-04-22T07:04:12.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.199+0000 7fda6ffff640 1 -- start start 2024-04-22T07:04:12.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.200+0000 7fda6ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda680dd610 0x7fda680def70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.200+0000 7fda6ffff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fda6804c650 0x7fda680df4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.200+0000 7fda6ffff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda6804cc00 con 0x7fda680d5e10 2024-04-22T07:04:12.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.200+0000 7fda6ffff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fda680092c0 con 0x7fda680dd610 2024-04-22T07:04:12.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.200+0000 7fda6ffff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fda6804e990 con 0x7fda6804c650 2024-04-22T07:04:12.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.201+0000 7fda6e7fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda680dd610 0x7fda680def70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.201+0000 7fda6e7fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda680dd610 0x7fda680def70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36274/0 (socket says 172.21.15.92:36274) 2024-04-22T07:04:12.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.201+0000 7fda6e7fc640 1 -- 172.21.15.92:0/2740817313 learned_addr learned my addr 172.21.15.92:0/2740817313 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.202+0000 7fda6e7fc640 1 -- 172.21.15.92:0/2740817313 >> v2:172.21.15.145:3300/0 conn(0x7fda6804c650 msgr2=0x7fda680df4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:04:12.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.202+0000 7fda6e7fc640 1 --2- 172.21.15.92:0/2740817313 >> v2:172.21.15.145:3300/0 conn(0x7fda6804c650 0x7fda680df4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.202+0000 7fda6e7fc640 1 -- 172.21.15.92:0/2740817313 >> v1:172.21.15.92:6789/0 conn(0x7fda680d5e10 legacy=0x7fda680dcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:12.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.202+0000 7fda6e7fc640 1 -- 172.21.15.92:0/2740817313 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda680086f0 con 0x7fda680dd610 2024-04-22T07:04:12.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.203+0000 7fda6e7fc640 1 --2- 172.21.15.92:0/2740817313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda680dd610 0x7fda680def70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda7006b100 tx=0x7fda700a9ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be6256cd5aefd5f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.204+0000 7fda6dffb640 1 -- 172.21.15.92:0/2740817313 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70074e60 con 0x7fda680dd610 2024-04-22T07:04:12.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.204+0000 7fda6dffb640 1 -- 172.21.15.92:0/2740817313 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda700ae070 con 0x7fda680dd610 2024-04-22T07:04:12.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.205+0000 7fda6ffff640 1 -- 172.21.15.92:0/2740817313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda680dd610 msgr2=0x7fda680def70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.205+0000 7fda6ffff640 1 --2- 172.21.15.92:0/2740817313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda680dd610 0x7fda680def70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda7006b100 tx=0x7fda700a9ef0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.206+0000 7fda6ffff640 1 -- 172.21.15.92:0/2740817313 shutdown_connections 2024-04-22T07:04:12.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.206+0000 7fda6ffff640 1 --2- 172.21.15.92:0/2740817313 >> v2:172.21.15.145:3300/0 conn(0x7fda6804c650 0x7fda680df4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.206+0000 7fda6ffff640 1 --2- 172.21.15.92:0/2740817313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda680dd610 0x7fda680def70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.206+0000 7fda6ffff640 1 -- 172.21.15.92:0/2740817313 >> 172.21.15.92:0/2740817313 conn(0x7fda6804dd20 msgr2=0x7fda6804e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.206+0000 7fda6ffff640 1 -- 172.21.15.92:0/2740817313 shutdown_connections 2024-04-22T07:04:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.206+0000 7fda6ffff640 1 -- 172.21.15.92:0/2740817313 wait complete. 2024-04-22T07:04:12.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.208+0000 7fda6ffff640 1 Processor -- start 2024-04-22T07:04:12.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.209+0000 7fda6ffff640 1 -- start start 2024-04-22T07:04:12.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.210+0000 7fda6ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda6804c650 0x7fda680ddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.210+0000 7fda6ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda680d5e10 0x7fda680cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.210+0000 7fda6ffff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda680dd610 0x7fda680516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.210+0000 7fda6ffff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fda68008f50 con 0x7fda6804c650 2024-04-22T07:04:12.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.210+0000 7fda6ffff640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fda68008c70 con 0x7fda680d5e10 2024-04-22T07:04:12.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.210+0000 7fda6ffff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fda6804d820 con 0x7fda680dd610 2024-04-22T07:04:12.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.210+0000 7fda6effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda6804c650 0x7fda680ddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.211+0000 7fda6e7fc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda680d5e10 0x7fda680cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.211+0000 7fda6e7fc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda680d5e10 0x7fda680cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:53900/0 (socket says 172.21.15.92:53900) 2024-04-22T07:04:12.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.211+0000 7fda6e7fc640 1 -- 172.21.15.92:0/3292114765 learned_addr learned my addr 172.21.15.92:0/3292114765 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.212+0000 7fda6e7fc640 1 -- 172.21.15.92:0/3292114765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda680dd610 msgr2=0x7fda680516d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:12.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.212+0000 7fda6e7fc640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda680dd610 0x7fda680516d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.212+0000 7fda6e7fc640 1 -- 172.21.15.92:0/3292114765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda6804c650 msgr2=0x7fda680ddaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.212+0000 7fda6e7fc640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda6804c650 0x7fda680ddaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.212+0000 7fda6e7fc640 1 -- 172.21.15.92:0/3292114765 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6804b660 con 0x7fda680d5e10 2024-04-22T07:04:12.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.212+0000 7fda6effd640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda6804c650 0x7fda680ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:12.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.213+0000 7fda6e7fc640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda680d5e10 0x7fda680cfa40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda7002db50 tx=0x7fda70076d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.213+0000 7fda57fff640 1 -- 172.21.15.92:0/3292114765 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda700c0020 con 0x7fda680d5e10 2024-04-22T07:04:12.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.213+0000 7fda6ffff640 1 -- 172.21.15.92:0/3292114765 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6804b940 con 0x7fda680d5e10 2024-04-22T07:04:12.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.213+0000 7fda6ffff640 1 -- 172.21.15.92:0/3292114765 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda680cf240 con 0x7fda680d5e10 2024-04-22T07:04:12.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.213+0000 7fda57fff640 1 -- 172.21.15.92:0/3292114765 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda700bc620 con 0x7fda680d5e10 2024-04-22T07:04:12.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.214+0000 7fda57fff640 1 -- 172.21.15.92:0/3292114765 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda700bc9d0 con 0x7fda680d5e10 2024-04-22T07:04:12.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.215+0000 7fda6ffff640 1 -- 172.21.15.92:0/3292114765 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda50005290 con 0x7fda680d5e10 2024-04-22T07:04:12.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.217+0000 7fda57fff640 1 -- 172.21.15.92:0/3292114765 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda700b0030 con 0x7fda680d5e10 2024-04-22T07:04:12.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.218+0000 7fda57fff640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fda5c06c9f0 0x7fda5c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.218+0000 7fda6effd640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fda5c06c9f0 0x7fda5c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.219+0000 7fda6effd640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fda5c06c9f0 0x7fda5c06eeb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fda60006fd0 tx=0x7fda60009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.219+0000 7fda57fff640 1 -- 172.21.15.92:0/3292114765 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fda70129ae0 con 0x7fda680d5e10 2024-04-22T07:04:12.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.220+0000 7fda57fff640 1 -- 172.21.15.92:0/3292114765 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda700f7e60 con 0x7fda680d5e10 2024-04-22T07:04:12.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.255+0000 7f5be3b59640 1 Processor -- start 2024-04-22T07:04:12.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.255+0000 7f5be3b59640 1 -- start start 2024-04-22T07:04:12.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.256+0000 7f5be3b59640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5bdc1577b0 0x7f5bdc06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.256+0000 7f5be3b59640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bdc0c6da0 con 0x7f5bdc06f040 2024-04-22T07:04:12.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.256+0000 7f5be3b59640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bdc0c7080 con 0x7f5bdc1510b0 2024-04-22T07:04:12.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.256+0000 7f5be3b59640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f5bdc0c7360 con 0x7f5bdc1577b0 2024-04-22T07:04:12.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.256+0000 7f583bba4640 1 Processor -- start 2024-04-22T07:04:12.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.257+0000 7f583bba4640 1 -- start start 2024-04-22T07:04:12.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.257+0000 7f5be18f5640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f5bdc1510b0 0x7f5bdc06fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:55222/0 (socket says 172.21.15.92:55222) 2024-04-22T07:04:12.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.257+0000 7f5be18f5640 1 -- 172.21.15.92:0/3109661364 learned_addr learned my addr 172.21.15.92:0/3109661364 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:12.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.257+0000 7f5be10f4640 1 --2- 172.21.15.92:0/3109661364 >> v2:172.21.15.145:3300/0 conn(0x7f5bdc1577b0 0x7f5bdc06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.258+0000 7f3f037bd640 1 Processor -- start 2024-04-22T07:04:12.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.258+0000 7f3f037bd640 1 -- start start 2024-04-22T07:04:12.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.258+0000 7f3bdc98e640 1 Processor -- start 2024-04-22T07:04:12.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.258+0000 7f3bdc98e640 1 -- start start 2024-04-22T07:04:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.258+0000 7f5be10f4640 1 -- 172.21.15.92:0/3109661364 >> v1:172.21.15.145:6789/0 conn(0x7f5bdc1510b0 legacy=0x7f5bdc06fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.258+0000 7f5be10f4640 1 -- 172.21.15.92:0/3109661364 >> v1:172.21.15.92:6789/0 conn(0x7f5bdc06f040 legacy=0x7f5bdc071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.258+0000 7f5be10f4640 1 -- 172.21.15.92:0/3109661364 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc0c7640 con 0x7f5bdc1577b0 2024-04-22T07:04:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f3f96fe1640 1 Processor -- start 2024-04-22T07:04:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f3f96fe1640 1 -- start start 2024-04-22T07:04:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f3bdc98e640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3bd00d92b0 0x7f3bd0038640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.258+0000 7f583bba4640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f582c0d8f30 0x7f582c045800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.258+0000 7f583bba4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c04b0d0 0x7f582c04b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.258+0000 7f583bba4640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f582c0df440 con 0x7f582c0dec90 2024-04-22T07:04:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.258+0000 7f583bba4640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f582c045d40 con 0x7f582c0d8f30 2024-04-22T07:04:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.258+0000 7f583bba4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f582c0463f0 con 0x7f582c04b0d0 2024-04-22T07:04:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f5839940640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f582c0d8f30 0x7f582c045800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f5839940640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f582c0d8f30 0x7f582c045800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36290/0 (socket says 172.21.15.92:36290) 2024-04-22T07:04:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f5839940640 1 -- 172.21.15.92:0/3597636372 learned_addr learned my addr 172.21.15.92:0/3597636372 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f583a141640 1 --2- 172.21.15.92:0/3597636372 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c04b0d0 0x7f582c04b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f3bdc98e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bd0037170 0x7f3bd0038b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f7214967640 1 Processor -- start 2024-04-22T07:04:12.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f3f037bd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3efc151040 0x7f3efc0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f3bdc98e640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bd00376d0 con 0x7f3bd00378f0 2024-04-22T07:04:12.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f5be10f4640 1 --2- 172.21.15.92:0/3109661364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc1577b0 0x7f5bdc06eb00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400aed0 tx=0x7f5bd402dea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=528d866248ae9b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3bd659c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bd0037170 0x7f3bd0038b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3bdc98e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f3bd00502c0 con 0x7f3bd0037170 2024-04-22T07:04:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3bdc98e640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3bd00505a0 con 0x7f3bd00d92b0 2024-04-22T07:04:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f037bd640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3efc15abd0 0x7f3efc0c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f5be08f3640 1 -- 172.21.15.92:0/3109661364 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd402e9c0 con 0x7f5bdc1577b0 2024-04-22T07:04:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3bd659c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bd0037170 0x7f3bd0038b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36294/0 (socket says 172.21.15.92:36294) 2024-04-22T07:04:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3bd659c640 1 -- 172.21.15.92:0/2872127818 learned_addr learned my addr 172.21.15.92:0/2872127818 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f037bd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3efc0c7160 0x7f3efc06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f5be08f3640 1 -- 172.21.15.92:0/3109661364 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd402eb60 con 0x7f5bdc1577b0 2024-04-22T07:04:12.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f037bd640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f3efc157760 con 0x7f3efc151040 2024-04-22T07:04:12.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f037bd640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3efc157bf0 con 0x7f3efc15abd0 2024-04-22T07:04:12.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f037bd640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3efc0cbca0 con 0x7f3efc0c7160 2024-04-22T07:04:12.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f01d5a640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3efc0c7160 0x7f3efc06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f5be08f3640 1 -- 172.21.15.92:0/3109661364 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4008440 con 0x7f5bdc1577b0 2024-04-22T07:04:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f583a141640 1 -- 172.21.15.92:0/3597636372 >> v2:172.21.15.92:3300/0 conn(0x7f582c0d8f30 msgr2=0x7f582c045800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f583a141640 1 --2- 172.21.15.92:0/3597636372 >> v2:172.21.15.92:3300/0 conn(0x7f582c0d8f30 0x7f582c045800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f583a141640 1 -- 172.21.15.92:0/3597636372 >> v1:172.21.15.92:6789/0 conn(0x7f582c0dec90 legacy=0x7f582c046880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.259+0000 7f583a141640 1 -- 172.21.15.92:0/3597636372 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582c0df720 con 0x7f582c04b0d0 2024-04-22T07:04:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f01d5a640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3efc0c7160 0x7f3efc06eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52504/0 (socket says 172.21.15.92:52504) 2024-04-22T07:04:12.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f01d5a640 1 -- 172.21.15.92:0/2185796689 learned_addr learned my addr 172.21.15.92:0/2185796689 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f7214967640 1 -- start start 2024-04-22T07:04:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3bd659c640 1 -- 172.21.15.92:0/2872127818 >> v2:172.21.15.145:3300/0 conn(0x7f3bd00d92b0 msgr2=0x7f3bd0038640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:04:12.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f583a141640 1 --2- 172.21.15.92:0/3597636372 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c04b0d0 0x7f582c04b520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f58240097a0 tx=0x7f582402b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb56b38104edb3a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f3f01559640 1 --2- 172.21.15.92:0/2185796689 >> v2:172.21.15.92:3300/0 conn(0x7f3efc151040 0x7f3efc0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f3f00d58640 1 --2- 172.21.15.92:0/2185796689 >> v2:172.21.15.145:3300/0 conn(0x7f3efc15abd0 0x7f3efc0c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f96fe1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f90152c10 0x7f3f90159d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f5be3b59640 1 -- 172.21.15.92:0/3109661364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc1577b0 msgr2=0x7f5bdc06eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f5be3b59640 1 --2- 172.21.15.92:0/3109661364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc1577b0 0x7f5bdc06eb00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400aed0 tx=0x7f5bd402dea0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f583893e640 1 -- 172.21.15.92:0/3597636372 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58240317f0 con 0x7f582c04b0d0 2024-04-22T07:04:12.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f96fe1640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f90159560 0x7f3f901631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f7214967640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7210159080 0x7f7210162e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f7214967640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f721006e0c0 0x7f721006e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f7214967640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f721006d340 con 0x7f721015ee80 2024-04-22T07:04:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f7214967640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f7210163360 con 0x7f7210159080 2024-04-22T07:04:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f7214967640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f72100c6d20 con 0x7f721006e0c0 2024-04-22T07:04:12.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f96fe1640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f90152ff0 con 0x7f3f900c6f20 2024-04-22T07:04:12.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f583893e640 1 -- 172.21.15.92:0/3597636372 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5824037070 con 0x7f582c04b0d0 2024-04-22T07:04:12.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f96fe1640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f3f9015a2a0 con 0x7f3f90152c10 2024-04-22T07:04:12.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3f96fe1640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3f900c65b0 con 0x7f3f90159560 2024-04-22T07:04:12.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f3f967e0640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f90159560 0x7f3f901631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f3f967e0640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f90159560 0x7f3f901631f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52506/0 (socket says 172.21.15.92:52506) 2024-04-22T07:04:12.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f3f967e0640 1 -- 172.21.15.92:0/1197892156 learned_addr learned my addr 172.21.15.92:0/1197892156 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f3f967e0640 1 -- 172.21.15.92:0/1197892156 >> v2:172.21.15.92:3300/0 conn(0x7f3f90152c10 msgr2=0x7f3f90159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f720e59c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7210159080 0x7f7210162e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f720e59c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7210159080 0x7f7210162e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:53928/0 (socket says 172.21.15.92:53928) 2024-04-22T07:04:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f720e59c640 1 -- 172.21.15.92:0/1278266690 learned_addr learned my addr 172.21.15.92:0/1278266690 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f3f95fdf640 1 --2- 172.21.15.92:0/1197892156 >> v2:172.21.15.92:3300/0 conn(0x7f3f90152c10 0x7f3f90159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f720dd9b640 1 --2- 172.21.15.92:0/1278266690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f721006e0c0 0x7f721006e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f4350831640 1 Processor -- start 2024-04-22T07:04:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f3f94fdd640 1 -- 172.21.15.92:0/1197892156 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3729712878 0 0) 0x7f3f90152ff0 con 0x7f3f900c6f20 2024-04-22T07:04:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f4350831640 1 -- start start 2024-04-22T07:04:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f5be3b59640 1 -- 172.21.15.92:0/3109661364 shutdown_connections 2024-04-22T07:04:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f5be3b59640 1 --2- 172.21.15.92:0/3109661364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc1577b0 0x7f5bdc06eb00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f5be3b59640 1 -- 172.21.15.92:0/3109661364 >> 172.21.15.92:0/3109661364 conn(0x7f5bdc0cc220 msgr2=0x7f5bdc0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3bd659c640 1 --2- 172.21.15.92:0/2872127818 >> v2:172.21.15.145:3300/0 conn(0x7f3bd00d92b0 0x7f3bd0038640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3bd659c640 1 -- 172.21.15.92:0/2872127818 >> v1:172.21.15.145:6789/0 conn(0x7f3bd00378f0 legacy=0x7f3bd00ded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:12.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.260+0000 7f3bd659c640 1 -- 172.21.15.92:0/2872127818 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd0050880 con 0x7f3bd0037170 2024-04-22T07:04:12.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f720dd9b640 1 -- 172.21.15.92:0/1278266690 >> v2:172.21.15.145:3300/0 conn(0x7f7210159080 msgr2=0x7f7210162e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:12.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f720dd9b640 1 --2- 172.21.15.92:0/1278266690 >> v2:172.21.15.145:3300/0 conn(0x7f7210159080 0x7f7210162e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f720dd9b640 1 -- 172.21.15.92:0/1278266690 >> v1:172.21.15.92:6789/0 conn(0x7f721015ee80 legacy=0x7f721015f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:12.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f5be3b59640 1 -- 172.21.15.92:0/3109661364 shutdown_connections 2024-04-22T07:04:12.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f3bd659c640 1 --2- 172.21.15.92:0/2872127818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd0037170 0x7f3bd0038b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0055360 tx=0x7f3bc802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46298940893efcb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f583bba4640 1 -- 172.21.15.92:0/3597636372 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c04b0d0 msgr2=0x7f582c04b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f583bba4640 1 --2- 172.21.15.92:0/3597636372 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c04b0d0 0x7f582c04b520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f58240097a0 tx=0x7f582402b890 comp rx=0 tx=0).stop 2024-04-22T07:04:12.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f583bba4640 1 -- 172.21.15.92:0/3597636372 shutdown_connections 2024-04-22T07:04:12.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f583bba4640 1 --2- 172.21.15.92:0/3597636372 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c04b0d0 0x7f582c04b520 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f583bba4640 1 --2- 172.21.15.92:0/3597636372 >> v2:172.21.15.92:3300/0 conn(0x7f582c0d8f30 0x7f582c045800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f583bba4640 1 -- 172.21.15.92:0/3597636372 >> 172.21.15.92:0/3597636372 conn(0x7f582c0505c0 msgr2=0x7f582c0509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f720dd9b640 1 -- 172.21.15.92:0/1278266690 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72100c6fe0 con 0x7f721006e0c0 2024-04-22T07:04:12.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f3f01559640 1 -- 172.21.15.92:0/2185796689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3efc0c7160 msgr2=0x7f3efc06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:12.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f3f01559640 1 --2- 172.21.15.92:0/2185796689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3efc0c7160 0x7f3efc06eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f3f01559640 1 -- 172.21.15.92:0/2185796689 >> v2:172.21.15.145:3300/0 conn(0x7f3efc15abd0 msgr2=0x7f3efc0c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:12.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f3f01559640 1 --2- 172.21.15.92:0/2185796689 >> v2:172.21.15.145:3300/0 conn(0x7f3efc15abd0 0x7f3efc0c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f3f01559640 1 -- 172.21.15.92:0/2185796689 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc156e30 con 0x7f3efc151040 2024-04-22T07:04:12.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f5be3b59640 1 -- 172.21.15.92:0/3109661364 wait complete. 2024-04-22T07:04:12.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f3f967e0640 1 --2- 172.21.15.92:0/1197892156 >> v2:172.21.15.92:3300/0 conn(0x7f3f90152c10 0x7f3f90159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f3f967e0640 1 -- 172.21.15.92:0/1197892156 >> v1:172.21.15.92:6789/0 conn(0x7f3f900c6f20 legacy=0x7f3f900cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.261+0000 7f3f967e0640 1 -- 172.21.15.92:0/1197892156 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f900c6870 con 0x7f3f90159560 2024-04-22T07:04:12.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f3f95fdf640 1 --2- 172.21.15.92:0/1197892156 >> v2:172.21.15.92:3300/0 conn(0x7f3f90152c10 0x7f3f90159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:12.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f4350831640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f434c154de0 0x7f434c15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f4350831640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c0caec0 0x7f434c0676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f583bba4640 1 -- 172.21.15.92:0/3597636372 shutdown_connections 2024-04-22T07:04:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f4350831640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f434c0c59b0 con 0x7f434c067bf0 2024-04-22T07:04:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f4350831640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f434c0c5c70 con 0x7f434c154de0 2024-04-22T07:04:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f4350831640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f434c0c5f50 con 0x7f434c0caec0 2024-04-22T07:04:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f4349d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c0caec0 0x7f434c0676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f4349d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c0caec0 0x7f434c0676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52532/0 (socket says 172.21.15.92:52532) 2024-04-22T07:04:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f4349d9b640 1 -- 172.21.15.92:0/3606788924 learned_addr learned my addr 172.21.15.92:0/3606788924 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f3bd5d9b640 1 -- 172.21.15.92:0/2872127818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8035020 con 0x7f3bd0037170 2024-04-22T07:04:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f3bd5d9b640 1 -- 172.21.15.92:0/2872127818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc8031890 con 0x7f3bd0037170 2024-04-22T07:04:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.262+0000 7f3bd5d9b640 1 -- 172.21.15.92:0/2872127818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8031c60 con 0x7f3bd0037170 2024-04-22T07:04:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f3bdc98e640 1 -- 172.21.15.92:0/2872127818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd0037170 msgr2=0x7f3bd0038b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f3bdc98e640 1 --2- 172.21.15.92:0/2872127818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd0037170 0x7f3bd0038b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0055360 tx=0x7f3bc802b930 comp rx=0 tx=0).stop 2024-04-22T07:04:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f720e59c640 1 --2- 172.21.15.92:0/1278266690 >> v2:172.21.15.145:3300/0 conn(0x7f7210159080 0x7f7210162e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3f01559640 1 --2- 172.21.15.92:0/2185796689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3efc151040 0x7f3efc0cb5f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0c63f0 tx=0x7f3eec034100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f687089e5d6155 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f4349d9b640 1 -- 172.21.15.92:0/3606788924 >> v2:172.21.15.145:3300/0 conn(0x7f434c154de0 msgr2=0x7f434c15b8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:04:12.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f4349d9b640 1 --2- 172.21.15.92:0/3606788924 >> v2:172.21.15.145:3300/0 conn(0x7f434c154de0 0x7f434c15b8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f4349d9b640 1 -- 172.21.15.92:0/3606788924 >> v1:172.21.15.92:6789/0 conn(0x7f434c067bf0 legacy=0x7f434c0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f4349d9b640 1 -- 172.21.15.92:0/3606788924 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434c0c6230 con 0x7f434c0caec0 2024-04-22T07:04:12.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f5be3b59640 1 Processor -- start 2024-04-22T07:04:12.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.263+0000 7f583bba4640 1 -- 172.21.15.92:0/3597636372 wait complete. 2024-04-22T07:04:12.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f720dd9b640 1 --2- 172.21.15.92:0/1278266690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f721006e0c0 0x7f721006e4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7208009d90 tx=0x7f720802bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b64545b822cfbc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f5be3b59640 1 -- start start 2024-04-22T07:04:12.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3ef3fff640 1 -- 172.21.15.92:0/2185796689 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec041070 con 0x7f3efc151040 2024-04-22T07:04:12.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3ef3fff640 1 -- 172.21.15.92:0/2185796689 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec03c440 con 0x7f3efc151040 2024-04-22T07:04:12.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3ef3fff640 1 -- 172.21.15.92:0/2185796689 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec03c850 con 0x7f3efc151040 2024-04-22T07:04:12.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3f967e0640 1 --2- 172.21.15.92:0/1197892156 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f90159560 0x7f3f901631f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f88014a00 tx=0x7f3f88034b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cafd245a36eb471 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3f94fdd640 1 -- 172.21.15.92:0/1197892156 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f880368d0 con 0x7f3f90159560 2024-04-22T07:04:12.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3f94fdd640 1 -- 172.21.15.92:0/1197892156 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f88036a70 con 0x7f3f90159560 2024-04-22T07:04:12.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3f94fdd640 1 -- 172.21.15.92:0/1197892156 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88036e40 con 0x7f3f90159560 2024-04-22T07:04:12.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3bdc98e640 1 -- 172.21.15.92:0/2872127818 shutdown_connections 2024-04-22T07:04:12.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3bdc98e640 1 --2- 172.21.15.92:0/2872127818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd0037170 0x7f3bd0038b80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3bdc98e640 1 --2- 172.21.15.92:0/2872127818 >> v2:172.21.15.145:3300/0 conn(0x7f3bd00d92b0 0x7f3bd0038640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3bdc98e640 1 -- 172.21.15.92:0/2872127818 >> 172.21.15.92:0/2872127818 conn(0x7f3bd0054600 msgr2=0x7f3bd0054a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3bdc98e640 1 -- 172.21.15.92:0/2872127818 shutdown_connections 2024-04-22T07:04:12.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f3bdc98e640 1 -- 172.21.15.92:0/2872127818 wait complete. 2024-04-22T07:04:12.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f720d59a640 1 -- 172.21.15.92:0/1278266690 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7208039070 con 0x7f721006e0c0 2024-04-22T07:04:12.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f720d59a640 1 -- 172.21.15.92:0/1278266690 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f720802cce0 con 0x7f721006e0c0 2024-04-22T07:04:12.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.264+0000 7f583bba4640 1 Processor -- start 2024-04-22T07:04:12.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.265+0000 7f583bba4640 1 -- start start 2024-04-22T07:04:12.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.265+0000 7f7214967640 1 -- 172.21.15.92:0/1278266690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f721006e0c0 msgr2=0x7f721006e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.265+0000 7f7214967640 1 --2- 172.21.15.92:0/1278266690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f721006e0c0 0x7f721006e4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7208009d90 tx=0x7f720802bf30 comp rx=0 tx=0).stop 2024-04-22T07:04:12.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.265+0000 7f3f037bd640 1 -- 172.21.15.92:0/2185796689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3efc151040 msgr2=0x7f3efc0cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.265+0000 7f3f96fe1640 1 -- 172.21.15.92:0/1197892156 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f90159560 msgr2=0x7f3f901631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.265+0000 7f3f96fe1640 1 --2- 172.21.15.92:0/1197892156 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f90159560 0x7f3f901631f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f88014a00 tx=0x7f3f88034b10 comp rx=0 tx=0).stop 2024-04-22T07:04:12.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.265+0000 7f7214967640 1 -- 172.21.15.92:0/1278266690 shutdown_connections 2024-04-22T07:04:12.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.265+0000 7f3f037bd640 1 --2- 172.21.15.92:0/2185796689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3efc151040 0x7f3efc0cb5f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0c63f0 tx=0x7f3eec034100 comp rx=0 tx=0).stop 2024-04-22T07:04:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.265+0000 7f7214967640 1 --2- 172.21.15.92:0/1278266690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f721006e0c0 0x7f721006e4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.265+0000 7f7214967640 1 --2- 172.21.15.92:0/1278266690 >> v2:172.21.15.145:3300/0 conn(0x7f7210159080 0x7f7210162e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.265+0000 7f7214967640 1 -- 172.21.15.92:0/1278266690 >> 172.21.15.92:0/1278266690 conn(0x7f721006bfd0 msgr2=0x7f721006f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.265+0000 7f4349d9b640 1 --2- 172.21.15.92:0/3606788924 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c0caec0 0x7f434c0676b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4340009870 tx=0x7f434002ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37d77316642c8495 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3f037bd640 1 -- 172.21.15.92:0/2185796689 shutdown_connections 2024-04-22T07:04:12.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3f037bd640 1 --2- 172.21.15.92:0/2185796689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3efc0c7160 0x7f3efc06eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3f96fe1640 1 -- 172.21.15.92:0/1197892156 shutdown_connections 2024-04-22T07:04:12.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3f96fe1640 1 --2- 172.21.15.92:0/1197892156 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f90159560 0x7f3f901631f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3f96fe1640 1 --2- 172.21.15.92:0/1197892156 >> v2:172.21.15.92:3300/0 conn(0x7f3f90152c10 0x7f3f90159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3f96fe1640 1 -- 172.21.15.92:0/1197892156 >> 172.21.15.92:0/1197892156 conn(0x7f3f9006b970 msgr2=0x7f3f9006bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be3b59640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc06f040 0x7f5bdc1f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be3b59640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bdc1510b0 0x7f5bdc0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3f037bd640 1 --2- 172.21.15.92:0/2185796689 >> v2:172.21.15.145:3300/0 conn(0x7f3efc15abd0 0x7f3efc0c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3f037bd640 1 --2- 172.21.15.92:0/2185796689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3efc151040 0x7f3efc0cb5f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3f037bd640 1 -- 172.21.15.92:0/2185796689 >> 172.21.15.92:0/2185796689 conn(0x7f3efc0cc220 msgr2=0x7f3efc0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be3b59640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5bdc1577b0 0x7f5bdc0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be3b59640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc070300 con 0x7f5bdc1510b0 2024-04-22T07:04:12.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be3b59640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc070180 con 0x7f5bdc06f040 2024-04-22T07:04:12.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be3b59640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5bdc0ccd40 con 0x7f5bdc1577b0 2024-04-22T07:04:12.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f583bba4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f582c04b0d0 0x7f582c0bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f583bba4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c0d8f30 0x7f582c0c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f583bba4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f582c0dec90 0x7f582c04ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f583bba4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f582c0062d0 con 0x7f582c0dec90 2024-04-22T07:04:12.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f583bba4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f582c046080 con 0x7f582c04b0d0 2024-04-22T07:04:12.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f583bba4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f582c0d9310 con 0x7f582c0d8f30 2024-04-22T07:04:12.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be10f4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bdc1510b0 0x7f5bdc0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3f037bd640 1 -- 172.21.15.92:0/2185796689 shutdown_connections 2024-04-22T07:04:12.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be10f4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bdc1510b0 0x7f5bdc0cb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36310/0 (socket says 172.21.15.92:36310) 2024-04-22T07:04:12.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be10f4640 1 -- 172.21.15.92:0/1231429930 learned_addr learned my addr 172.21.15.92:0/1231429930 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3f037bd640 1 -- 172.21.15.92:0/2185796689 wait complete. 2024-04-22T07:04:12.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f434959a640 1 -- 172.21.15.92:0/3606788924 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434002f850 con 0x7f434c0caec0 2024-04-22T07:04:12.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f434959a640 1 -- 172.21.15.92:0/3606788924 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f434002f9f0 con 0x7f434c0caec0 2024-04-22T07:04:12.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f434959a640 1 -- 172.21.15.92:0/3606788924 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434002fdc0 con 0x7f434c0caec0 2024-04-22T07:04:12.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3bdc98e640 1 Processor -- start 2024-04-22T07:04:12.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be10f4640 1 -- 172.21.15.92:0/1231429930 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5bdc1577b0 msgr2=0x7f5bdc0cb5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:12.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be10f4640 1 --2- 172.21.15.92:0/1231429930 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5bdc1577b0 0x7f5bdc0cb5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be10f4640 1 -- 172.21.15.92:0/1231429930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc06f040 msgr2=0x7f5bdc1f89c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:12.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be10f4640 1 --2- 172.21.15.92:0/1231429930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc06f040 0x7f5bdc1f89c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be10f4640 1 -- 172.21.15.92:0/1231429930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc1f93f0 con 0x7f5bdc1510b0 2024-04-22T07:04:12.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f583913f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c0d8f30 0x7f582c0c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f583913f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c0d8f30 0x7f582c0c1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52546/0 (socket says 172.21.15.92:52546) 2024-04-22T07:04:12.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f583913f640 1 -- 172.21.15.92:0/3637776897 learned_addr learned my addr 172.21.15.92:0/3637776897 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f5be10f4640 1 --2- 172.21.15.92:0/1231429930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bdc1510b0 0x7f5bdc0cb0b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000f80 tx=0x7f5bd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f583913f640 1 -- 172.21.15.92:0/3637776897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f582c04b0d0 msgr2=0x7f582c0bcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f583913f640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f582c04b0d0 0x7f582c0bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f583913f640 1 -- 172.21.15.92:0/3637776897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f582c0dec90 msgr2=0x7f582c04ef40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:12.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f5bd27fc640 1 -- 172.21.15.92:0/1231429930 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4040420 con 0x7f5bdc1510b0 2024-04-22T07:04:12.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f5be3b59640 1 -- 172.21.15.92:0/1231429930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bdc1f9690 con 0x7f5bdc1510b0 2024-04-22T07:04:12.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f4350831640 1 -- 172.21.15.92:0/3606788924 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c0caec0 msgr2=0x7f434c0676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f4350831640 1 --2- 172.21.15.92:0/3606788924 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c0caec0 0x7f434c0676b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4340009870 tx=0x7f434002ba30 comp rx=0 tx=0).stop 2024-04-22T07:04:12.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f5bd27fc640 1 -- 172.21.15.92:0/1231429930 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd4045070 con 0x7f5bdc1510b0 2024-04-22T07:04:12.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f5be3b59640 1 -- 172.21.15.92:0/1231429930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bdc0c6520 con 0x7f5bdc1510b0 2024-04-22T07:04:12.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f3bdc98e640 1 -- start start 2024-04-22T07:04:12.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f583a141640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f582c0dec90 0x7f582c04ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f3bdc98e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd0037170 0x7f3bd00d9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f583913f640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f582c0dec90 0x7f582c04ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f3bdc98e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bd00378f0 0x7f3bd0175200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f3bdc98e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00d92b0 0x7f3bd00dbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f583913f640 1 -- 172.21.15.92:0/3637776897 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582c0c9150 con 0x7f582c0d8f30 2024-04-22T07:04:12.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.267+0000 7f583a141640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f582c0dec90 0x7f582c04ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:12.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f583913f640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c0d8f30 0x7f582c0c1e60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f583000c430 tx=0x7f583000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f3bd659c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bd00378f0 0x7f3bd0175200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f3f037bd640 1 Processor -- start 2024-04-22T07:04:12.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f4350831640 1 -- 172.21.15.92:0/3606788924 shutdown_connections 2024-04-22T07:04:12.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f4350831640 1 --2- 172.21.15.92:0/3606788924 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c0caec0 0x7f434c0676b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f5bd27fc640 1 -- 172.21.15.92:0/1231429930 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4040800 con 0x7f5bdc1510b0 2024-04-22T07:04:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f4350831640 1 --2- 172.21.15.92:0/3606788924 >> v2:172.21.15.145:3300/0 conn(0x7f434c154de0 0x7f434c15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f4350831640 1 -- 172.21.15.92:0/3606788924 >> 172.21.15.92:0/3606788924 conn(0x7f434c073b60 msgr2=0x7f434c0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f3f037bd640 1 -- start start 2024-04-22T07:04:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f3f96fe1640 1 -- 172.21.15.92:0/1197892156 shutdown_connections 2024-04-22T07:04:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f5be3b59640 1 -- 172.21.15.92:0/1231429930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc4005290 con 0x7f5bdc1510b0 2024-04-22T07:04:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f582a7fc640 1 -- 172.21.15.92:0/3637776897 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5830017070 con 0x7f582c0d8f30 2024-04-22T07:04:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f582a7fc640 1 -- 172.21.15.92:0/3637776897 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58300052d0 con 0x7f582c0d8f30 2024-04-22T07:04:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f582a7fc640 1 -- 172.21.15.92:0/3637776897 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5830003810 con 0x7f582c0d8f30 2024-04-22T07:04:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f583bba4640 1 -- 172.21.15.92:0/3637776897 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582c0c9430 con 0x7f582c0d8f30 2024-04-22T07:04:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f583bba4640 1 -- 172.21.15.92:0/3637776897 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f582c04e460 con 0x7f582c0d8f30 2024-04-22T07:04:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f7214967640 1 -- 172.21.15.92:0/1278266690 shutdown_connections 2024-04-22T07:04:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.266+0000 7f7214967640 1 -- 172.21.15.92:0/1278266690 wait complete. 2024-04-22T07:04:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f7214967640 1 Processor -- start 2024-04-22T07:04:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f7214967640 1 -- start start 2024-04-22T07:04:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f3f037bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3efc0c7160 0x7f3efc151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f3f037bd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3efc151040 0x7f3efc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f4350831640 1 -- 172.21.15.92:0/3606788924 shutdown_connections 2024-04-22T07:04:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f4350831640 1 -- 172.21.15.92:0/3606788924 wait complete. 2024-04-22T07:04:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3f01559640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3efc0c7160 0x7f3efc151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3f01559640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3efc0c7160 0x7f3efc151bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36352/0 (socket says 172.21.15.92:36352) 2024-04-22T07:04:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3f01559640 1 -- 172.21.15.92:0/4152097782 learned_addr learned my addr 172.21.15.92:0/4152097782 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3f037bd640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3efc15abd0 0x7f3efc1e1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f3f96fe1640 1 -- 172.21.15.92:0/1197892156 wait complete. 2024-04-22T07:04:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3f96fe1640 1 Processor -- start 2024-04-22T07:04:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f7214967640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721006e0c0 0x7f721014ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f7214967640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7210159080 0x7f72101ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f7214967640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f721015ee80 0x7f7210149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f3bd659c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bd00378f0 0x7f3bd0175200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52554/0 (socket says 172.21.15.92:52554) 2024-04-22T07:04:12.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f3bd659c640 1 -- 172.21.15.92:0/3708131405 learned_addr learned my addr 172.21.15.92:0/3708131405 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f3bd6d9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd0037170 0x7f3bd00d9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd00df4a0 con 0x7f3bd00d92b0 2024-04-22T07:04:12.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd0038120 con 0x7f3bd0037170 2024-04-22T07:04:12.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd00df620 con 0x7f3bd00378f0 2024-04-22T07:04:12.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f3bd659c640 1 -- 172.21.15.92:0/3708131405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd0037170 msgr2=0x7f3bd00d9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f7214967640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f721006d8c0 con 0x7f721006e0c0 2024-04-22T07:04:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f7214967640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f721006d5e0 con 0x7f7210159080 2024-04-22T07:04:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f7214967640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f721006dba0 con 0x7f721015ee80 2024-04-22T07:04:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f720e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721006e0c0 0x7f721014ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f720e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721006e0c0 0x7f721014ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36340/0 (socket says 172.21.15.92:36340) 2024-04-22T07:04:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f720e59c640 1 -- 172.21.15.92:0/3821716324 learned_addr learned my addr 172.21.15.92:0/3821716324 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f720e59c640 1 -- 172.21.15.92:0/3821716324 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f721015ee80 msgr2=0x7f7210149880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f720e59c640 1 --2- 172.21.15.92:0/3821716324 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f721015ee80 0x7f7210149880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f720e59c640 1 -- 172.21.15.92:0/3821716324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7210159080 msgr2=0x7f72101ea7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f720e59c640 1 --2- 172.21.15.92:0/3821716324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7210159080 0x7f72101ea7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f720e59c640 1 -- 172.21.15.92:0/3821716324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721014b290 con 0x7f721006e0c0 2024-04-22T07:04:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f720e59c640 1 --2- 172.21.15.92:0/3821716324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721006e0c0 0x7f721014ad50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f720000d910 tx=0x7f720000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f3bd659c640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd0037170 0x7f3bd00d9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f3bd659c640 1 -- 172.21.15.92:0/3708131405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00d92b0 msgr2=0x7f3bd00dbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.268+0000 7f3bd759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00d92b0 0x7f3bd00dbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3bd659c640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00d92b0 0x7f3bd00dbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3bd659c640 1 -- 172.21.15.92:0/3708131405 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd0175740 con 0x7f3bd00378f0 2024-04-22T07:04:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3bd759e640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00d92b0 0x7f3bd00dbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3f01d5a640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3efc15abd0 0x7f3efc1e1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc06fe00 con 0x7f3efc0c7160 2024-04-22T07:04:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1573f0 con 0x7f3efc151040 2024-04-22T07:04:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3f00d58640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3efc151040 0x7f3efc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3efc1ed7d0 con 0x7f3efc15abd0 2024-04-22T07:04:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f3bd659c640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bd00378f0 0x7f3bd0175200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3bc802b970 tx=0x7f3bc80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.269+0000 7f5bd27fc640 1 -- 172.21.15.92:0/1231429930 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bd402ed10 con 0x7f5bdc1510b0 2024-04-22T07:04:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f5bd27fc640 1 --2- 172.21.15.92:0/1231429930 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5bc006c9c0 0x7f5bc006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f5be18f5640 1 --2- 172.21.15.92:0/1231429930 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5bc006c9c0 0x7f5bc006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.270+0000 7f5bd27fc640 1 -- 172.21.15.92:0/1231429930 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5bd40ae160 con 0x7f5bdc1510b0 2024-04-22T07:04:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f5be18f5640 1 --2- 172.21.15.92:0/1231429930 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5bc006c9c0 0x7f5bc006ee80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8001710 tx=0x7f5bd8007c80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f3bbf7fe640 1 -- 172.21.15.92:0/3708131405 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8040070 con 0x7f3bd00378f0 2024-04-22T07:04:12.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f3bbf7fe640 1 -- 172.21.15.92:0/3708131405 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc803a420 con 0x7f3bd00378f0 2024-04-22T07:04:12.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd00d9a40 con 0x7f3bd00378f0 2024-04-22T07:04:12.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f583bba4640 1 -- 172.21.15.92:0/3637776897 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f582c0057d0 con 0x7f582c0d8f30 2024-04-22T07:04:12.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd004bf30 con 0x7f3bd00378f0 2024-04-22T07:04:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f3f01d5a640 1 -- 172.21.15.92:0/4152097782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3efc151040 msgr2=0x7f3efc06eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f3f01d5a640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3efc151040 0x7f3efc06eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f3f01d5a640 1 -- 172.21.15.92:0/4152097782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3efc0c7160 msgr2=0x7f3efc151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f3f01d5a640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3efc0c7160 0x7f3efc151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f3f01d5a640 1 -- 172.21.15.92:0/4152097782 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc1ed270 con 0x7f3efc15abd0 2024-04-22T07:04:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f3f01559640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3efc0c7160 0x7f3efc151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f582a7fc640 1 -- 172.21.15.92:0/3637776897 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f583001a070 con 0x7f582c0d8f30 2024-04-22T07:04:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f3f01d5a640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3efc15abd0 0x7f3efc1e1ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0715f0 tx=0x7f3ef400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f4350831640 1 Processor -- start 2024-04-22T07:04:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f3bbf7fe640 1 -- 172.21.15.92:0/3708131405 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8035040 con 0x7f3bd00378f0 2024-04-22T07:04:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f4350831640 1 -- start start 2024-04-22T07:04:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.271+0000 7f3f96fe1640 1 -- start start 2024-04-22T07:04:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f582a7fc640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f581806c9c0 0x7f581806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f582a7fc640 1 -- 172.21.15.92:0/3637776897 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f583008ae20 con 0x7f582c0d8f30 2024-04-22T07:04:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f5839940640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f581806c9c0 0x7f581806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f3f96fe1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f900c6f20 0x7f3f901487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f3f96fe1640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f90152c10 0x7f3f900c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f3f96fe1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f90159560 0x7f3f90157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f3f96fe1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90159040 con 0x7f3f900c6f20 2024-04-22T07:04:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f3f96fe1640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90158d60 con 0x7f3f90159560 2024-04-22T07:04:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f3f96fe1640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3f900c74a0 con 0x7f3f90152c10 2024-04-22T07:04:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f3f967e0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f90159560 0x7f3f90157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f3f967e0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f90159560 0x7f3f90157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:53962/0 (socket says 172.21.15.92:53962) 2024-04-22T07:04:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.272+0000 7f3f967e0640 1 -- 172.21.15.92:0/4151579361 learned_addr learned my addr 172.21.15.92:0/4151579361 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f3f95fdf640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f900c6f20 0x7f3f901487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f5839940640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f581806c9c0 0x7f581806ee80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5834076410 tx=0x7f583406e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f3f967e0640 1 -- 172.21.15.92:0/4151579361 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f90152c10 msgr2=0x7f3f900c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f3f967e0640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f90152c10 0x7f3f900c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f3f967e0640 1 -- 172.21.15.92:0/4151579361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f900c6f20 msgr2=0x7f3f901487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f3f967e0640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f900c6f20 0x7f3f901487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f3f967e0640 1 -- 172.21.15.92:0/4151579361 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f900c8110 con 0x7f3f90159560 2024-04-22T07:04:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f4350831640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434c067bf0 0x7f434c0b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f4350831640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434c0caec0 0x7f434c1f1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f3f95fdf640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f900c6f20 0x7f3f901487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:12.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f4350831640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c154de0 0x7f434c14fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f4350831640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1551c0 con 0x7f434c067bf0 2024-04-22T07:04:12.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f4350831640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f434c0ccfb0 con 0x7f434c0caec0 2024-04-22T07:04:12.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f4350831640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f434c06c7b0 con 0x7f434c154de0 2024-04-22T07:04:12.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f434a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434c067bf0 0x7f434c0b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f434a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434c067bf0 0x7f434c0b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36372/0 (socket says 172.21.15.92:36372) 2024-04-22T07:04:12.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f434a59c640 1 -- 172.21.15.92:0/1496449654 learned_addr learned my addr 172.21.15.92:0/1496449654 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:12.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f3f967e0640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f90159560 0x7f3f90157d70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f88035120 tx=0x7f3f88004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb8005290 con 0x7f3bd00378f0 2024-04-22T07:04:12.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f4349d9b640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434c0caec0 0x7f434c1f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f434ad9d640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c154de0 0x7f434c14fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f3f867fc640 1 -- 172.21.15.92:0/4151579361 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88011810 con 0x7f3f90159560 2024-04-22T07:04:12.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.274+0000 7f3f867fc640 1 -- 172.21.15.92:0/4151579361 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f88046070 con 0x7f3f90159560 2024-04-22T07:04:12.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f434a59c640 1 -- 172.21.15.92:0/1496449654 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c154de0 msgr2=0x7f434c14fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f434a59c640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c154de0 0x7f434c14fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f434a59c640 1 -- 172.21.15.92:0/1496449654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434c0caec0 msgr2=0x7f434c1f1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f434a59c640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434c0caec0 0x7f434c1f1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.273+0000 7f434a59c640 1 -- 172.21.15.92:0/1496449654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434c150b80 con 0x7f434c067bf0 2024-04-22T07:04:12.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.274+0000 7f3f867fc640 1 -- 172.21.15.92:0/4151579361 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8800f050 con 0x7f3f90159560 2024-04-22T07:04:12.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.274+0000 7f3f96fe1640 1 -- 172.21.15.92:0/4151579361 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f90158620 con 0x7f3f90159560 2024-04-22T07:04:12.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.274+0000 7f434ad9d640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c154de0 0x7f434c14fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:12.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.274+0000 7f3f96fe1640 1 -- 172.21.15.92:0/4151579361 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f900c8690 con 0x7f3f90159560 2024-04-22T07:04:12.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.274+0000 7f434a59c640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434c067bf0 0x7f434c0b2d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f434400ba80 tx=0x7f434400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.275+0000 7f3f96fe1640 1 -- 172.21.15.92:0/4151579361 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f90147dd0 con 0x7f3f90159560 2024-04-22T07:04:12.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.276+0000 7f3f867fc640 1 -- 172.21.15.92:0/4151579361 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f88036900 con 0x7f3f90159560 2024-04-22T07:04:12.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.276+0000 7f3f867fc640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3f7406c8f0 0x7f3f7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.276+0000 7f3ef1ffb640 1 -- 172.21.15.92:0/4152097782 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400cc50 con 0x7f3efc15abd0 2024-04-22T07:04:12.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.276+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc14dca0 con 0x7f3efc15abd0 2024-04-22T07:04:12.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.276+0000 7f3f867fc640 1 -- 172.21.15.92:0/4151579361 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f880aee90 con 0x7f3f90159560 2024-04-22T07:04:12.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.276+0000 7f582a7fc640 1 -- 172.21.15.92:0/3637776897 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5830058be0 con 0x7f582c0d8f30 2024-04-22T07:04:12.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.276+0000 7f3f95fdf640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3f7406c8f0 0x7f3f7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.277+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1ed4f0 con 0x7f3efc15abd0 2024-04-22T07:04:12.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.277+0000 7f3ef1ffb640 1 -- 172.21.15.92:0/4152097782 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef400cdf0 con 0x7f3efc15abd0 2024-04-22T07:04:12.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.277+0000 7f3ef1ffb640 1 -- 172.21.15.92:0/4152097782 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4003b90 con 0x7f3efc15abd0 2024-04-22T07:04:12.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.277+0000 7f3f95fdf640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3f7406c8f0 0x7f3f7406edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00ab40 tx=0x7f3f8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.277+0000 7f71feffd640 1 -- 172.21.15.92:0/3821716324 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200004030 con 0x7f721006e0c0 2024-04-22T07:04:12.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.277+0000 7f71feffd640 1 -- 172.21.15.92:0/3821716324 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72000041d0 con 0x7f721006e0c0 2024-04-22T07:04:12.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.277+0000 7f71feffd640 1 -- 172.21.15.92:0/3821716324 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200009780 con 0x7f721006e0c0 2024-04-22T07:04:12.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.277+0000 7f7214967640 1 -- 172.21.15.92:0/3821716324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721014d010 con 0x7f721006e0c0 2024-04-22T07:04:12.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.277+0000 7f433affd640 1 -- 172.21.15.92:0/1496449654 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344019070 con 0x7f434c067bf0 2024-04-22T07:04:12.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.277+0000 7f433affd640 1 -- 172.21.15.92:0/1496449654 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f434400cc90 con 0x7f434c067bf0 2024-04-22T07:04:12.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.277+0000 7f433affd640 1 -- 172.21.15.92:0/1496449654 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344014680 con 0x7f434c067bf0 2024-04-22T07:04:12.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.278+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3edc005290 con 0x7f3efc15abd0 2024-04-22T07:04:12.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.278+0000 7f4350831640 1 -- 172.21.15.92:0/1496449654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434c150e60 con 0x7f434c067bf0 2024-04-22T07:04:12.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.278+0000 7f4350831640 1 -- 172.21.15.92:0/1496449654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434c14ff40 con 0x7f434c067bf0 2024-04-22T07:04:12.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.278+0000 7f3bbf7fe640 1 -- 172.21.15.92:0/3708131405 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3bc8038020 con 0x7f3bd00378f0 2024-04-22T07:04:12.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.277+0000 7f7214967640 1 -- 172.21.15.92:0/3821716324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721014a2c0 con 0x7f721006e0c0 2024-04-22T07:04:12.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.279+0000 7f3bbf7fe640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3bc406c9c0 0x7f3bc406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.279+0000 7f3bd6d9d640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3bc406c9c0 0x7f3bc406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.279+0000 7f3ef1ffb640 1 -- 172.21.15.92:0/4152097782 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ef4012050 con 0x7f3efc15abd0 2024-04-22T07:04:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.279+0000 7f71fcff9640 1 -- 172.21.15.92:0/3821716324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72100c26b0 con 0x7f721006e0c0 2024-04-22T07:04:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.279+0000 7f3bbf7fe640 1 -- 172.21.15.92:0/3708131405 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3bc80ae480 con 0x7f3bd00378f0 2024-04-22T07:04:12.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.278+0000 7f5bd27fc640 1 -- 172.21.15.92:0/1231429930 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5bd407c6e0 con 0x7f5bdc1510b0 2024-04-22T07:04:12.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.280+0000 7f3ef1ffb640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3ee006c920 0x7f3ee006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.280+0000 7f3bd6d9d640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3bc406c9c0 0x7f3bc406ee80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0098c0 tx=0x7f3bcc009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.279+0000 7f4338ff9640 1 -- 172.21.15.92:0/1496449654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4324005290 con 0x7f434c067bf0 2024-04-22T07:04:12.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.280+0000 7f3f01559640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3ee006c920 0x7f3ee006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.281+0000 7f3f01559640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3ee006c920 0x7f3ee006ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3eec03c5c0 tx=0x7f3eec038000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.282+0000 7f71feffd640 1 -- 172.21.15.92:0/3821716324 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7200009920 con 0x7f721006e0c0 2024-04-22T07:04:12.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.282+0000 7f71feffd640 1 --2- 172.21.15.92:0/3821716324 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f71f006c9c0 0x7f71f006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.282+0000 7f433affd640 1 -- 172.21.15.92:0/1496449654 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f434401c070 con 0x7f434c067bf0 2024-04-22T07:04:12.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.282+0000 7f720dd9b640 1 --2- 172.21.15.92:0/3821716324 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f71f006c9c0 0x7f71f006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.282+0000 7f71feffd640 1 -- 172.21.15.92:0/3821716324 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f720008c740 con 0x7f721006e0c0 2024-04-22T07:04:12.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.283+0000 7f433affd640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f432c06c940 0x7f432c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.283+0000 7f433affd640 1 -- 172.21.15.92:0/1496449654 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f434408bf40 con 0x7f434c067bf0 2024-04-22T07:04:12.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.283+0000 7f720dd9b640 1 --2- 172.21.15.92:0/3821716324 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f71f006c9c0 0x7f71f006ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7208036000 tx=0x7f7208030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.283+0000 7f4349d9b640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f432c06c940 0x7f432c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.284+0000 7f4349d9b640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f432c06c940 0x7f432c06ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4340036000 tx=0x7f4340008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.286+0000 7f71feffd640 1 -- 172.21.15.92:0/3821716324 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72100c26b0 con 0x7f721006e0c0 2024-04-22T07:04:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.286+0000 7f3f867fc640 1 -- 172.21.15.92:0/4151579361 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f8807d230 con 0x7f3f90159560 2024-04-22T07:04:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.287+0000 7f3ef1ffb640 1 -- 172.21.15.92:0/4152097782 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3ef4057d00 con 0x7f3efc15abd0 2024-04-22T07:04:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.287+0000 7f3bbf7fe640 1 -- 172.21.15.92:0/3708131405 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3bc802d040 con 0x7f3bd00378f0 2024-04-22T07:04:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.291+0000 7f3ef1ffb640 1 -- 172.21.15.92:0/4152097782 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3efc1ed270 con 0x7f3efc15abd0 2024-04-22T07:04:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.294+0000 7f433affd640 1 -- 172.21.15.92:0/1496449654 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4344023020 con 0x7f434c067bf0 2024-04-22T07:04:12.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.634+0000 7fda6ffff640 1 -- 172.21.15.92:0/3292114765 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fda50005850 con 0x7fda680d5e10 2024-04-22T07:04:12.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.634+0000 7fda57fff640 1 -- 172.21.15.92:0/3292114765 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fda700f7760 con 0x7fda680d5e10 2024-04-22T07:04:12.632 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:12.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.638+0000 7fda55ffb640 1 -- 172.21.15.92:0/3292114765 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fda5c06c9f0 msgr2=0x7fda5c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.638+0000 7fda55ffb640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fda5c06c9f0 0x7fda5c06eeb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fda60006fd0 tx=0x7fda60009060 comp rx=0 tx=0).stop 2024-04-22T07:04:12.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.638+0000 7fda55ffb640 1 -- 172.21.15.92:0/3292114765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda680d5e10 msgr2=0x7fda680cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.638+0000 7fda55ffb640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda680d5e10 0x7fda680cfa40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda7002db50 tx=0x7fda70076d70 comp rx=0 tx=0).stop 2024-04-22T07:04:12.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.638+0000 7fda55ffb640 1 -- 172.21.15.92:0/3292114765 shutdown_connections 2024-04-22T07:04:12.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.638+0000 7fda55ffb640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fda5c06c9f0 0x7fda5c06eeb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.638+0000 7fda55ffb640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda680dd610 0x7fda680516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.638+0000 7fda55ffb640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda680d5e10 0x7fda680cfa40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.638+0000 7fda55ffb640 1 --2- 172.21.15.92:0/3292114765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda6804c650 0x7fda680ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.638+0000 7fda55ffb640 1 -- 172.21.15.92:0/3292114765 >> 172.21.15.92:0/3292114765 conn(0x7fda6804dd20 msgr2=0x7fda680d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.639+0000 7fda55ffb640 1 -- 172.21.15.92:0/3292114765 shutdown_connections 2024-04-22T07:04:12.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.639+0000 7fda55ffb640 1 -- 172.21.15.92:0/3292114765 wait complete. 2024-04-22T07:04:12.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-04-22T07:04:12.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.674+0000 7f3f96fe1640 1 -- 172.21.15.92:0/4151579361 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3f9015bc40 con 0x7f3f90159560 2024-04-22T07:04:12.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.675+0000 7f3f867fc640 1 -- 172.21.15.92:0/4151579361 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3f8807cb30 con 0x7f3f90159560 2024-04-22T07:04:12.673 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2024-04-22T07:04:12.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.677+0000 7f3f96fe1640 1 -- 172.21.15.92:0/4151579361 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3f7406c8f0 msgr2=0x7f3f7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.677+0000 7f3f96fe1640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3f7406c8f0 0x7f3f7406edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00ab40 tx=0x7f3f8c008040 comp rx=0 tx=0).stop 2024-04-22T07:04:12.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.677+0000 7f3f96fe1640 1 -- 172.21.15.92:0/4151579361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f90159560 msgr2=0x7f3f90157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.677+0000 7f3f96fe1640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f90159560 0x7f3f90157d70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f88035120 tx=0x7f3f88004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:12.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.677+0000 7f3f96fe1640 1 -- 172.21.15.92:0/4151579361 shutdown_connections 2024-04-22T07:04:12.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.677+0000 7f3f96fe1640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3f7406c8f0 0x7f3f7406edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.677+0000 7f3f96fe1640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f90159560 0x7f3f90157d70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.677+0000 7f3f96fe1640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f90152c10 0x7f3f900c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.677+0000 7f3f96fe1640 1 --2- 172.21.15.92:0/4151579361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f900c6f20 0x7f3f901487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.677+0000 7f3f96fe1640 1 -- 172.21.15.92:0/4151579361 >> 172.21.15.92:0/4151579361 conn(0x7f3f9006b970 msgr2=0x7f3f90151910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.677+0000 7f3f96fe1640 1 -- 172.21.15.92:0/4151579361 shutdown_connections 2024-04-22T07:04:12.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.677+0000 7f3f96fe1640 1 -- 172.21.15.92:0/4151579361 wait complete. 2024-04-22T07:04:12.686 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-04-22T07:04:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.695+0000 7f583bba4640 1 -- 172.21.15.92:0/3637776897 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f582c004a10 con 0x7f582c0d8f30 2024-04-22T07:04:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.695+0000 7f582a7fc640 1 -- 172.21.15.92:0/3637776897 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f583005cee0 con 0x7f582c0d8f30 2024-04-22T07:04:12.693 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2024-04-22T07:04:12.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.695+0000 7f5be3b59640 1 -- 172.21.15.92:0/1231429930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5bc4005580 con 0x7f5bdc1510b0 2024-04-22T07:04:12.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.696+0000 7f5bd27fc640 1 -- 172.21.15.92:0/1231429930 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5bd407bfe0 con 0x7f5bdc1510b0 2024-04-22T07:04:12.694 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.697+0000 7f583bba4640 1 -- 172.21.15.92:0/3637776897 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f581806c9c0 msgr2=0x7f581806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.697+0000 7f583bba4640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f581806c9c0 0x7f581806ee80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5834076410 tx=0x7f583406e040 comp rx=0 tx=0).stop 2024-04-22T07:04:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.697+0000 7f583bba4640 1 -- 172.21.15.92:0/3637776897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c0d8f30 msgr2=0x7f582c0c1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.697+0000 7f583bba4640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c0d8f30 0x7f582c0c1e60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f583000c430 tx=0x7f583000c8f0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.697+0000 7f583bba4640 1 -- 172.21.15.92:0/3637776897 shutdown_connections 2024-04-22T07:04:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.697+0000 7f583bba4640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f581806c9c0 0x7f581806ee80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.697+0000 7f583bba4640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f582c0dec90 0x7f582c04ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.697+0000 7f583bba4640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f582c0d8f30 0x7f582c0c1e60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.697+0000 7f583bba4640 1 --2- 172.21.15.92:0/3637776897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f582c04b0d0 0x7f582c0bcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.697+0000 7f583bba4640 1 -- 172.21.15.92:0/3637776897 >> 172.21.15.92:0/3637776897 conn(0x7f582c0505c0 msgr2=0x7f582c046780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.698+0000 7f583bba4640 1 -- 172.21.15.92:0/3637776897 shutdown_connections 2024-04-22T07:04:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.698+0000 7f583bba4640 1 -- 172.21.15.92:0/3637776897 wait complete. 2024-04-22T07:04:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.699+0000 7f5be3b59640 1 -- 172.21.15.92:0/1231429930 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5bc006c9c0 msgr2=0x7f5bc006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.699+0000 7f5be3b59640 1 --2- 172.21.15.92:0/1231429930 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5bc006c9c0 0x7f5bc006ee80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8001710 tx=0x7f5bd8007c80 comp rx=0 tx=0).stop 2024-04-22T07:04:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.699+0000 7f5be3b59640 1 -- 172.21.15.92:0/1231429930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bdc1510b0 msgr2=0x7f5bdc0cb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.699+0000 7f5be3b59640 1 --2- 172.21.15.92:0/1231429930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bdc1510b0 0x7f5bdc0cb0b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000f80 tx=0x7f5bd4004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.699+0000 7f5be3b59640 1 -- 172.21.15.92:0/1231429930 shutdown_connections 2024-04-22T07:04:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.699+0000 7f5be3b59640 1 --2- 172.21.15.92:0/1231429930 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5bc006c9c0 0x7f5bc006ee80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.699+0000 7f5be3b59640 1 --2- 172.21.15.92:0/1231429930 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5bdc1577b0 0x7f5bdc0cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.699+0000 7f5be3b59640 1 --2- 172.21.15.92:0/1231429930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bdc1510b0 0x7f5bdc0cb0b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.699+0000 7f5be3b59640 1 --2- 172.21.15.92:0/1231429930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc06f040 0x7f5bdc1f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.699+0000 7f5be3b59640 1 -- 172.21.15.92:0/1231429930 >> 172.21.15.92:0/1231429930 conn(0x7f5bdc0cc220 msgr2=0x7f5bdc14b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.699+0000 7f5be3b59640 1 -- 172.21.15.92:0/1231429930 shutdown_connections 2024-04-22T07:04:12.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.700+0000 7f5be3b59640 1 -- 172.21.15.92:0/1231429930 wait complete. 2024-04-22T07:04:12.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.703+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3bb8005850 con 0x7f3bd00378f0 2024-04-22T07:04:12.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.704+0000 7f3bbf7fe640 1 -- 172.21.15.92:0/3708131405 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3bc807c1b0 con 0x7f3bd00378f0 2024-04-22T07:04:12.702 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:12.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.707+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3bc406c9c0 msgr2=0x7f3bc406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.707+0000 7f3bdc98e640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3bc406c9c0 0x7f3bc406ee80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0098c0 tx=0x7f3bcc009060 comp rx=0 tx=0).stop 2024-04-22T07:04:12.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.707+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bd00378f0 msgr2=0x7f3bd0175200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.707+0000 7f3bdc98e640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bd00378f0 0x7f3bd0175200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3bc802b970 tx=0x7f3bc80046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.707+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 shutdown_connections 2024-04-22T07:04:12.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.707+0000 7f3bdc98e640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3bc406c9c0 0x7f3bc406ee80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.707+0000 7f3bdc98e640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bd00d92b0 0x7f3bd00dbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.707+0000 7f3bdc98e640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bd00378f0 0x7f3bd0175200 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.707+0000 7f3bdc98e640 1 --2- 172.21.15.92:0/3708131405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd0037170 0x7f3bd00d9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.707+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 >> 172.21.15.92:0/3708131405 conn(0x7f3bd0054600 msgr2=0x7f3bd00549e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.707+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 shutdown_connections 2024-04-22T07:04:12.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.707+0000 7f3bdc98e640 1 -- 172.21.15.92:0/3708131405 wait complete. 2024-04-22T07:04:12.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-04-22T07:04:12.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.713+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3edc005580 con 0x7f3efc15abd0 2024-04-22T07:04:12.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.713+0000 7f71fcff9640 1 -- 172.21.15.92:0/3821716324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f721015d6f0 con 0x7f721006e0c0 2024-04-22T07:04:12.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.713+0000 7f71feffd640 1 -- 172.21.15.92:0/3821716324 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f721015d6f0 con 0x7f721006e0c0 2024-04-22T07:04:12.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.713+0000 7f3ef1ffb640 1 -- 172.21.15.92:0/4152097782 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ef4010040 con 0x7f3efc15abd0 2024-04-22T07:04:12.712 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2024-04-22T07:04:12.712 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2024-04-22T07:04:12.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-04-22T07:04:12.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.716+0000 7f71fcff9640 1 -- 172.21.15.92:0/3821716324 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f71f006c9c0 msgr2=0x7f71f006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.716+0000 7f71fcff9640 1 --2- 172.21.15.92:0/3821716324 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f71f006c9c0 0x7f71f006ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7208036000 tx=0x7f7208030000 comp rx=0 tx=0).stop 2024-04-22T07:04:12.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.716+0000 7f71fcff9640 1 -- 172.21.15.92:0/3821716324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721006e0c0 msgr2=0x7f721014ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.716+0000 7f71fcff9640 1 --2- 172.21.15.92:0/3821716324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721006e0c0 0x7f721014ad50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f720000d910 tx=0x7f720000ddd0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.716+0000 7f71fcff9640 1 -- 172.21.15.92:0/3821716324 shutdown_connections 2024-04-22T07:04:12.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.716+0000 7f71fcff9640 1 --2- 172.21.15.92:0/3821716324 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f71f006c9c0 0x7f71f006ee80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.716+0000 7f71fcff9640 1 --2- 172.21.15.92:0/3821716324 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f721015ee80 0x7f7210149880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.716+0000 7f71fcff9640 1 --2- 172.21.15.92:0/3821716324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7210159080 0x7f72101ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.716+0000 7f71fcff9640 1 --2- 172.21.15.92:0/3821716324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721006e0c0 0x7f721014ad50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.716+0000 7f71fcff9640 1 -- 172.21.15.92:0/3821716324 >> 172.21.15.92:0/3821716324 conn(0x7f721006bfd0 msgr2=0x7f72100717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f71fcff9640 1 -- 172.21.15.92:0/3821716324 shutdown_connections 2024-04-22T07:04:12.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f71fcff9640 1 -- 172.21.15.92:0/3821716324 wait complete. 2024-04-22T07:04:12.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3ee006c920 msgr2=0x7f3ee006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f3f037bd640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3ee006c920 0x7f3ee006ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3eec03c5c0 tx=0x7f3eec038000 comp rx=0 tx=0).stop 2024-04-22T07:04:12.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3efc15abd0 msgr2=0x7f3efc1e1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f3f037bd640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3efc15abd0 0x7f3efc1e1ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0715f0 tx=0x7f3ef400ef90 comp rx=0 tx=0).stop 2024-04-22T07:04:12.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 shutdown_connections 2024-04-22T07:04:12.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f3f037bd640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f3ee006c920 0x7f3ee006ede0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f3f037bd640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3efc15abd0 0x7f3efc1e1ac0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f3f037bd640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3efc151040 0x7f3efc06eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f3f037bd640 1 --2- 172.21.15.92:0/4152097782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3efc0c7160 0x7f3efc151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 >> 172.21.15.92:0/4152097782 conn(0x7f3efc0cc220 msgr2=0x7f3efc14ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 shutdown_connections 2024-04-22T07:04:12.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.717+0000 7f3f037bd640 1 -- 172.21.15.92:0/4152097782 wait complete. 2024-04-22T07:04:12.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.720+0000 7f4338ff9640 1 -- 172.21.15.92:0/1496449654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4324005850 con 0x7f434c067bf0 2024-04-22T07:04:12.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.721+0000 7f433affd640 1 -- 172.21.15.92:0/1496449654 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4344023020 con 0x7f434c067bf0 2024-04-22T07:04:12.718 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2024-04-22T07:04:12.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-04-22T07:04:12.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.724+0000 7f4338ff9640 1 -- 172.21.15.92:0/1496449654 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f432c06c940 msgr2=0x7f432c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.724+0000 7f4338ff9640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f432c06c940 0x7f432c06ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4340036000 tx=0x7f4340008030 comp rx=0 tx=0).stop 2024-04-22T07:04:12.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.724+0000 7f4338ff9640 1 -- 172.21.15.92:0/1496449654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434c067bf0 msgr2=0x7f434c0b2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.724+0000 7f4338ff9640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434c067bf0 0x7f434c0b2d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f434400ba80 tx=0x7f434400bf40 comp rx=0 tx=0).stop 2024-04-22T07:04:12.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.724+0000 7f4338ff9640 1 -- 172.21.15.92:0/1496449654 shutdown_connections 2024-04-22T07:04:12.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.724+0000 7f4338ff9640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f432c06c940 0x7f432c06ee00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.724+0000 7f4338ff9640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f434c154de0 0x7f434c14fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.724+0000 7f4338ff9640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434c0caec0 0x7f434c1f1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.724+0000 7f4338ff9640 1 --2- 172.21.15.92:0/1496449654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f434c067bf0 0x7f434c0b2d90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.724+0000 7f4338ff9640 1 -- 172.21.15.92:0/1496449654 >> 172.21.15.92:0/1496449654 conn(0x7f434c073b60 msgr2=0x7f434c0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.724+0000 7f4338ff9640 1 -- 172.21.15.92:0/1496449654 shutdown_connections 2024-04-22T07:04:12.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:12.724+0000 7f4338ff9640 1 -- 172.21.15.92:0/1496449654 wait complete. 2024-04-22T07:04:12.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-04-22T07:04:12.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2024-04-22T07:04:12.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2024-04-22T07:04:13.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-22T07:04:13.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-22T07:04:13.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-22T07:04:13.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-22T07:04:13.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-22T07:04:13.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-22T07:04:13.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-22T07:04:13.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-22T07:04:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.784+0000 7f991cb38640 1 Processor -- start 2024-04-22T07:04:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.784+0000 7f991cb38640 1 -- start start 2024-04-22T07:04:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.785+0000 7f991cb38640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f99180700f0 0x7f991806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.785+0000 7f991cb38640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9918071c90 con 0x7f991806f5d0 2024-04-22T07:04:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.785+0000 7f991cb38640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9918165350 con 0x7f991806eb00 2024-04-22T07:04:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.785+0000 7f991cb38640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f99180c85c0 con 0x7f99180700f0 2024-04-22T07:04:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.786+0000 7f9916d9d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f991806f5d0 0x7f99181721d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46718/0 (socket says 172.21.15.92:46718) 2024-04-22T07:04:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.786+0000 7f9915d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f99180700f0 0x7f991806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.786+0000 7f9916d9d640 1 -- 172.21.15.92:0/741774136 learned_addr learned my addr 172.21.15.92:0/741774136 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:13.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.787+0000 7f9915d9b640 1 -- 172.21.15.92:0/741774136 >> v1:172.21.15.145:6789/0 conn(0x7f991806eb00 legacy=0x7f99180724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.787+0000 7f9915d9b640 1 -- 172.21.15.92:0/741774136 >> v1:172.21.15.92:6789/0 conn(0x7f991806f5d0 legacy=0x7f99181721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.787+0000 7f9915d9b640 1 -- 172.21.15.92:0/741774136 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99180c87f0 con 0x7f99180700f0 2024-04-22T07:04:13.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.788+0000 7f9915d9b640 1 --2- 172.21.15.92:0/741774136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99180700f0 0x7f991806eee0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9910006570 tx=0x7f9910030060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c10fd9eaed978e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.789+0000 7f991559a640 1 -- 172.21.15.92:0/741774136 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910030e10 con 0x7f99180700f0 2024-04-22T07:04:13.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.789+0000 7f991559a640 1 -- 172.21.15.92:0/741774136 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9910003990 con 0x7f99180700f0 2024-04-22T07:04:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.791+0000 7f991cb38640 1 -- 172.21.15.92:0/741774136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99180700f0 msgr2=0x7f991806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.791+0000 7f991cb38640 1 --2- 172.21.15.92:0/741774136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99180700f0 0x7f991806eee0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9910006570 tx=0x7f9910030060 comp rx=0 tx=0).stop 2024-04-22T07:04:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.791+0000 7f991cb38640 1 -- 172.21.15.92:0/741774136 shutdown_connections 2024-04-22T07:04:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.791+0000 7f991cb38640 1 --2- 172.21.15.92:0/741774136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99180700f0 0x7f991806eee0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.791+0000 7f991cb38640 1 -- 172.21.15.92:0/741774136 >> 172.21.15.92:0/741774136 conn(0x7f99180cc220 msgr2=0x7f99180cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.791+0000 7f991cb38640 1 -- 172.21.15.92:0/741774136 shutdown_connections 2024-04-22T07:04:13.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.791+0000 7f991cb38640 1 -- 172.21.15.92:0/741774136 wait complete. 2024-04-22T07:04:13.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.793+0000 7f991cb38640 1 Processor -- start 2024-04-22T07:04:13.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.794+0000 7f991cb38640 1 -- start start 2024-04-22T07:04:13.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.795+0000 7f991cb38640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f991806eb00 0x7f9918158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.795+0000 7f991cb38640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f991806f5d0 0x7f991815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.795+0000 7f991cb38640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99180b9550 0x7f99180c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.795+0000 7f991cb38640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f991806cca0 con 0x7f99180b9550 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.795+0000 7f991cb38640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9918071f50 con 0x7f991806f5d0 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.795+0000 7f991cb38640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9918168380 con 0x7f991806eb00 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.795+0000 7f9916d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99180b9550 0x7f99180c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.795+0000 7f9916d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99180b9550 0x7f99180c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36384/0 (socket says 172.21.15.92:36384) 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.795+0000 7f9916d9d640 1 -- 172.21.15.92:0/3782591514 learned_addr learned my addr 172.21.15.92:0/3782591514 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.796+0000 7f9916d9d640 1 -- 172.21.15.92:0/3782591514 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f991806eb00 msgr2=0x7f9918158b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.796+0000 7f9916d9d640 1 --2- 172.21.15.92:0/3782591514 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f991806eb00 0x7f9918158b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.796+0000 7f9916d9d640 1 -- 172.21.15.92:0/3782591514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f991806f5d0 msgr2=0x7f991815c100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.796+0000 7f9916d9d640 1 --2- 172.21.15.92:0/3782591514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f991806f5d0 0x7f991815c100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.796+0000 7f9916d9d640 1 -- 172.21.15.92:0/3782591514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991815cfe0 con 0x7f99180b9550 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.797+0000 7f9916d9d640 1 --2- 172.21.15.92:0/3782591514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99180b9550 0x7f99180c5520 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f990c00b890 tx=0x7f990c00bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.798+0000 7f9906ffd640 1 -- 172.21.15.92:0/3782591514 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c00c920 con 0x7f99180b9550 2024-04-22T07:04:13.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.798+0000 7f991cb38640 1 -- 172.21.15.92:0/3782591514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991815c740 con 0x7f99180b9550 2024-04-22T07:04:13.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.798+0000 7f991cb38640 1 -- 172.21.15.92:0/3782591514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991815cd00 con 0x7f99180b9550 2024-04-22T07:04:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.798+0000 7f9906ffd640 1 -- 172.21.15.92:0/3782591514 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c010070 con 0x7f99180b9550 2024-04-22T07:04:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.798+0000 7f9906ffd640 1 -- 172.21.15.92:0/3782591514 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c00cd40 con 0x7f99180b9550 2024-04-22T07:04:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.801+0000 7f991cb38640 1 -- 172.21.15.92:0/3782591514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98fc005290 con 0x7f99180b9550 2024-04-22T07:04:13.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.808+0000 7f9906ffd640 1 -- 172.21.15.92:0/3782591514 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f990c006070 con 0x7f99180b9550 2024-04-22T07:04:13.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.808+0000 7f9906ffd640 1 --2- 172.21.15.92:0/3782591514 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f98f806c9c0 0x7f98f806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.809+0000 7f9906ffd640 1 -- 172.21.15.92:0/3782591514 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f990c08b9a0 con 0x7f99180b9550 2024-04-22T07:04:13.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.809+0000 7f991659c640 1 --2- 172.21.15.92:0/3782591514 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f98f806c9c0 0x7f98f806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.810+0000 7f991659c640 1 --2- 172.21.15.92:0/3782591514 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f98f806c9c0 0x7f98f806ee80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99080014f0 tx=0x7f9908007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.811+0000 7f9906ffd640 1 -- 172.21.15.92:0/3782591514 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f990c059e40 con 0x7f99180b9550 2024-04-22T07:04:13.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.856+0000 7f4dce5fa640 1 Processor -- start 2024-04-22T07:04:13.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.856+0000 7f4dce5fa640 1 -- start start 2024-04-22T07:04:13.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.857+0000 7f4dce5fa640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 0x7f4dc0039e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.857+0000 7f4dce5fa640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4dc004ffd0 0x7f4dc00dfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.857+0000 7f4dce5fa640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dc003a4b0 con 0x7f4dc00e0490 2024-04-22T07:04:13.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.857+0000 7f4dce5fa640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4dc00dffe0 con 0x7f4dc004ffd0 2024-04-22T07:04:13.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.857+0000 7f4dce5fa640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc0050460 con 0x7f4dc00d9250 2024-04-22T07:04:13.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.858+0000 7f4dccdf7640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4dc00e0490 0x7f4dc00071b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46726/0 (socket says 172.21.15.92:46726) 2024-04-22T07:04:13.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.858+0000 7f4dccdf7640 1 -- 172.21.15.92:0/3284561931 learned_addr learned my addr 172.21.15.92:0/3284561931 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.859+0000 7f4dcd5f8640 1 --2- 172.21.15.92:0/3284561931 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 0x7f4dc0039e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.859+0000 7f4dcddf9640 1 --2- 172.21.15.92:0/3284561931 >> v2:172.21.15.145:3300/0 conn(0x7f4dc004ffd0 0x7f4dc00dfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.859+0000 7f4dcddf9640 1 -- 172.21.15.92:0/3284561931 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 msgr2=0x7f4dc0039e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.859+0000 7f4dcddf9640 1 --2- 172.21.15.92:0/3284561931 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 0x7f4dc0039e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.859+0000 7f4dcddf9640 1 -- 172.21.15.92:0/3284561931 >> v1:172.21.15.92:6789/0 conn(0x7f4dc00e0490 legacy=0x7f4dc00071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.860+0000 7f4dcddf9640 1 -- 172.21.15.92:0/3284561931 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc004ef30 con 0x7f4dc004ffd0 2024-04-22T07:04:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.860+0000 7f4dcd5f8640 1 --2- 172.21.15.92:0/3284561931 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 0x7f4dc0039e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.861+0000 7f4dcddf9640 1 --2- 172.21.15.92:0/3284561931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4dc004ffd0 0x7f4dc00dfaa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4db80097a0 tx=0x7f4db802b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e24385ebf514908 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.861+0000 7f4dbffff640 1 -- 172.21.15.92:0/3284561931 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8037070 con 0x7f4dc004ffd0 2024-04-22T07:04:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.862+0000 7f4dbffff640 1 -- 172.21.15.92:0/3284561931 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db8030710 con 0x7f4dc004ffd0 2024-04-22T07:04:13.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.863+0000 7f4dce5fa640 1 -- 172.21.15.92:0/3284561931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4dc004ffd0 msgr2=0x7f4dc00dfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.863+0000 7f4dce5fa640 1 --2- 172.21.15.92:0/3284561931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4dc004ffd0 0x7f4dc00dfaa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4db80097a0 tx=0x7f4db802b770 comp rx=0 tx=0).stop 2024-04-22T07:04:13.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.863+0000 7f4dce5fa640 1 -- 172.21.15.92:0/3284561931 shutdown_connections 2024-04-22T07:04:13.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.863+0000 7f4dce5fa640 1 --2- 172.21.15.92:0/3284561931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4dc004ffd0 0x7f4dc00dfaa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.863+0000 7f4dce5fa640 1 --2- 172.21.15.92:0/3284561931 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 0x7f4dc0039e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.863+0000 7f4dce5fa640 1 -- 172.21.15.92:0/3284561931 >> 172.21.15.92:0/3284561931 conn(0x7f4dc000e160 msgr2=0x7f4dc000e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:13.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.863+0000 7f4dce5fa640 1 -- 172.21.15.92:0/3284561931 shutdown_connections 2024-04-22T07:04:13.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.864+0000 7f4dce5fa640 1 -- 172.21.15.92:0/3284561931 wait complete. 2024-04-22T07:04:13.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.865+0000 7f4dce5fa640 1 Processor -- start 2024-04-22T07:04:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.866+0000 7f4dce5fa640 1 -- start start 2024-04-22T07:04:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.867+0000 7f4dce5fa640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4dc004ffd0 0x7f4dc00df0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.867+0000 7f4dce5fa640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 0x7f4dc0038fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.867+0000 7f4dce5fa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc00e0490 0x7f4dc0039500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.867+0000 7f4dce5fa640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc004f7d0 con 0x7f4dc00e0490 2024-04-22T07:04:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.867+0000 7f4dce5fa640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc004f4f0 con 0x7f4dc004ffd0 2024-04-22T07:04:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.867+0000 7f4dce5fa640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc004fab0 con 0x7f4dc00d9250 2024-04-22T07:04:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.867+0000 7f4dccdf7640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 0x7f4dc0038fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.867+0000 7f4dccdf7640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 0x7f4dc0038fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52600/0 (socket says 172.21.15.92:52600) 2024-04-22T07:04:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.867+0000 7f4dccdf7640 1 -- 172.21.15.92:0/2055101119 learned_addr learned my addr 172.21.15.92:0/2055101119 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.868+0000 7f4dccdf7640 1 -- 172.21.15.92:0/2055101119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4dc004ffd0 msgr2=0x7f4dc00df0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T07:04:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.868+0000 7f4dccdf7640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4dc004ffd0 0x7f4dc00df0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.868+0000 7f4dccdf7640 1 -- 172.21.15.92:0/2055101119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc00e0490 msgr2=0x7f4dc0039500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.868+0000 7f4dcddf9640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc00e0490 0x7f4dc0039500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.868+0000 7f4dccdf7640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc00e0490 0x7f4dc0039500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.868+0000 7f4dccdf7640 1 -- 172.21.15.92:0/2055101119 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc00dfdf0 con 0x7f4dc00d9250 2024-04-22T07:04:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.869+0000 7f4dcddf9640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc00e0490 0x7f4dc0039500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.869+0000 7f4dccdf7640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 0x7f4dc0038fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4dc806b920 tx=0x7f4dc80695e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.869+0000 7f4dbdffb640 1 -- 172.21.15.92:0/2055101119 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8090070 con 0x7f4dc00d9250 2024-04-22T07:04:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.869+0000 7f4dce5fa640 1 -- 172.21.15.92:0/2055101119 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc00df830 con 0x7f4dc00d9250 2024-04-22T07:04:13.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.869+0000 7f4dce5fa640 1 -- 172.21.15.92:0/2055101119 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc00384e0 con 0x7f4dc00d9250 2024-04-22T07:04:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.871+0000 7f4dce5fa640 1 -- 172.21.15.92:0/2055101119 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc00d48b0 con 0x7f4dc00d9250 2024-04-22T07:04:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.871+0000 7f4dbdffb640 1 -- 172.21.15.92:0/2055101119 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc804fe20 con 0x7f4dc00d9250 2024-04-22T07:04:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.871+0000 7f4dbdffb640 1 -- 172.21.15.92:0/2055101119 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80947d0 con 0x7f4dc00d9250 2024-04-22T07:04:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.874+0000 7f4dbdffb640 1 -- 172.21.15.92:0/2055101119 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4dc8093070 con 0x7f4dc00d9250 2024-04-22T07:04:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.874+0000 7f4dbdffb640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4db406ca40 0x7f4db406ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.874+0000 7f4dbdffb640 1 -- 172.21.15.92:0/2055101119 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4dc8107a90 con 0x7f4dc00d9250 2024-04-22T07:04:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.875+0000 7f4dcd5f8640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4db406ca40 0x7f4db406ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.875+0000 7f4dcd5f8640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4db406ca40 0x7f4db406ef00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009560 tx=0x7f4dc4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.877+0000 7f4dbdffb640 1 -- 172.21.15.92:0/2055101119 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4dc00d48b0 con 0x7f4dc00d9250 2024-04-22T07:04:13.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.912+0000 7f9dc03b6640 1 Processor -- start 2024-04-22T07:04:13.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.912+0000 7f9dc03b6640 1 -- start start 2024-04-22T07:04:13.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.914+0000 7f9dc03b6640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9db81510b0 0x7f9db806fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.914+0000 7f9dc03b6640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db80c6da0 con 0x7f9db81577b0 2024-04-22T07:04:13.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.914+0000 7f9dc03b6640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db80c7080 con 0x7f9db806ffb0 2024-04-22T07:04:13.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.914+0000 7f9dc03b6640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f9db80c7360 con 0x7f9db81510b0 2024-04-22T07:04:13.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.914+0000 7f9dbd951640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9db81577b0 0x7f9db806eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46742/0 (socket says 172.21.15.92:46742) 2024-04-22T07:04:13.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.914+0000 7f9dbd951640 1 -- 172.21.15.92:0/39591172 learned_addr learned my addr 172.21.15.92:0/39591172 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:13.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.914+0000 7f9dbe152640 1 --2- 172.21.15.92:0/39591172 >> v2:172.21.15.92:3300/0 conn(0x7f9db81510b0 0x7f9db806fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.915+0000 7f9dbe152640 1 -- 172.21.15.92:0/39591172 >> v1:172.21.15.145:6789/0 conn(0x7f9db806ffb0 legacy=0x7f9db8071ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:04:13.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.915+0000 7f9dbe152640 1 -- 172.21.15.92:0/39591172 >> v1:172.21.15.92:6789/0 conn(0x7f9db81577b0 legacy=0x7f9db806eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.915+0000 7f9dbe152640 1 -- 172.21.15.92:0/39591172 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db80c7640 con 0x7f9db81510b0 2024-04-22T07:04:13.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.916+0000 7f9dbe152640 1 --2- 172.21.15.92:0/39591172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9db81510b0 0x7f9db806fa70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009a30 tx=0x7f9db402b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae5a1cfe8327c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.917+0000 7f9dbd150640 1 -- 172.21.15.92:0/39591172 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4034070 con 0x7f9db81510b0 2024-04-22T07:04:13.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.917+0000 7f9dbd150640 1 -- 172.21.15.92:0/39591172 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db4030620 con 0x7f9db81510b0 2024-04-22T07:04:13.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.918+0000 7f9dc03b6640 1 -- 172.21.15.92:0/39591172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9db81510b0 msgr2=0x7f9db806fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.918+0000 7f9dc03b6640 1 --2- 172.21.15.92:0/39591172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9db81510b0 0x7f9db806fa70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009a30 tx=0x7f9db402b9f0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.918+0000 7f9dc03b6640 1 -- 172.21.15.92:0/39591172 shutdown_connections 2024-04-22T07:04:13.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.918+0000 7f9dc03b6640 1 --2- 172.21.15.92:0/39591172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9db81510b0 0x7f9db806fa70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.918+0000 7f9dc03b6640 1 -- 172.21.15.92:0/39591172 >> 172.21.15.92:0/39591172 conn(0x7f9db80cc220 msgr2=0x7f9db80cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:13.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.919+0000 7f9dc03b6640 1 -- 172.21.15.92:0/39591172 shutdown_connections 2024-04-22T07:04:13.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.919+0000 7f64cb542640 1 Processor -- start 2024-04-22T07:04:13.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.919+0000 7f9dc03b6640 1 -- 172.21.15.92:0/39591172 wait complete. 2024-04-22T07:04:13.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.920+0000 7f64cb542640 1 -- start start 2024-04-22T07:04:13.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.921+0000 7f9dc03b6640 1 Processor -- start 2024-04-22T07:04:13.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.921+0000 7f64cb542640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f64bc004af0 0x7f64bc0d4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.921+0000 7f64cb542640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64bc04b740 0x7f64bc045710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f64cb542640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64bc0460e0 con 0x7f64bc045c50 2024-04-22T07:04:13.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f64cb542640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f64bc007a90 con 0x7f64bc004af0 2024-04-22T07:04:13.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f64cb542640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f64bc0d5530 con 0x7f64bc04b740 2024-04-22T07:04:13.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.921+0000 7f905e4e4640 1 Processor -- start 2024-04-22T07:04:13.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.921+0000 7f905e4e4640 1 -- start start 2024-04-22T07:04:13.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.921+0000 7f9dc03b6640 1 -- start start 2024-04-22T07:04:13.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f9dc03b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9db806ffb0 0x7f9db81f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f9dc03b6640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9db81510b0 0x7f9db81593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f9dc03b6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9db81577b0 0x7f9db8151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f9dc03b6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9db806f390 con 0x7f9db806ffb0 2024-04-22T07:04:13.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f9dc03b6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9db806f210 con 0x7f9db81577b0 2024-04-22T07:04:13.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f9dc03b6640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9db80723f0 con 0x7f9db81510b0 2024-04-22T07:04:13.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f64cad41640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f64bc045c50 0x7f64bc0de3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:55236/0 (socket says 172.21.15.92:55236) 2024-04-22T07:04:13.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f64cad41640 1 -- 172.21.15.92:0/3580027820 learned_addr learned my addr 172.21.15.92:0/3580027820 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:13.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f9dbd951640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9db81510b0 0x7f9db81593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f9dbe953640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9db81577b0 0x7f9db8151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f9dbd951640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9db81510b0 0x7f9db81593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52626/0 (socket says 172.21.15.92:52626) 2024-04-22T07:04:13.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f9dbd951640 1 -- 172.21.15.92:0/4240702392 learned_addr learned my addr 172.21.15.92:0/4240702392 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:13.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f905e4e4640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f90580c63d0 0x7f905815f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f905e4e4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f905815f790 0x7f90580c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f905e4e4640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90581627b0 con 0x7f90581592e0 2024-04-22T07:04:13.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f905e4e4640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f905815fc40 con 0x7f90580c63d0 2024-04-22T07:04:13.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f905e4e4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9058162a90 con 0x7f905815f790 2024-04-22T07:04:13.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f64c9d3f640 1 --2- 172.21.15.92:0/3580027820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64bc04b740 0x7f64bc045710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.922+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1451496789 0 0) 0x7f64bc0460e0 con 0x7f64bc045c50 2024-04-22T07:04:13.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64b0003520 con 0x7f64bc045c50 2024-04-22T07:04:13.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f9dbe152640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9db806ffb0 0x7f9db81f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f905ca81640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f905815f790 0x7f90580c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f905ca81640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f905815f790 0x7f90580c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52636/0 (socket says 172.21.15.92:52636) 2024-04-22T07:04:13.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f905ca81640 1 -- 172.21.15.92:0/1761929638 learned_addr learned my addr 172.21.15.92:0/1761929638 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:13.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f90577fe640 1 --2- 172.21.15.92:0/1761929638 >> v2:172.21.15.145:3300/0 conn(0x7f90580c63d0 0x7f905815f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f9dbe953640 1 -- 172.21.15.92:0/4240702392 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9db81510b0 msgr2=0x7f9db81593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f9dbe953640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9db81510b0 0x7f9db81593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f9dbe953640 1 -- 172.21.15.92:0/4240702392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9db806ffb0 msgr2=0x7f9db81f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f9dbe953640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9db806ffb0 0x7f9db81f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f9dbe953640 1 -- 172.21.15.92:0/4240702392 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db81f4b00 con 0x7f9db81577b0 2024-04-22T07:04:13.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f9dbe152640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9db806ffb0 0x7f9db81f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:13.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64ca540640 1 --2- 172.21.15.92:0/3580027820 >> v2:172.21.15.92:3300/0 conn(0x7f64bc004af0 0x7f64bc0d4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 <== mon.1 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3179879779 0 0) 0x7f64b0003520 con 0x7f64bc045c50 2024-04-22T07:04:13.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64bc04b740 msgr2=0x7f64bc045710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f441da38640 1 Processor -- start 2024-04-22T07:04:13.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 --2- 172.21.15.92:0/3580027820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64bc04b740 0x7f64bc045710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 >> v2:172.21.15.92:3300/0 conn(0x7f64bc004af0 msgr2=0x7f64bc0d4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 --2- 172.21.15.92:0/3580027820 >> v2:172.21.15.92:3300/0 conn(0x7f64bc004af0 0x7f64bc0d4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f7637fff640 1 Processor -- start 2024-04-22T07:04:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f441da38640 1 -- start start 2024-04-22T07:04:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc006e80 con 0x7f64bc045c50 2024-04-22T07:04:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 <== mon.1 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f64c40743a0 con 0x7f64bc045c50 2024-04-22T07:04:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 >> v1:172.21.15.145:6789/0 conn(0x7f64bc045c50 legacy=0x7f64bc0de3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f90577fe640 1 -- 172.21.15.92:0/1761929638 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f905815f790 msgr2=0x7f90580c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f90577fe640 1 --2- 172.21.15.92:0/1761929638 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f905815f790 0x7f90580c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f90577fe640 1 -- 172.21.15.92:0/1761929638 >> v1:172.21.15.145:6789/0 conn(0x7f90581592e0 legacy=0x7f905806eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f90577fe640 1 -- 172.21.15.92:0/1761929638 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9058162d70 con 0x7f90580c63d0 2024-04-22T07:04:13.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 --2- 172.21.15.92:0/3580027820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64b00039b0 0x7f64b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 --2- 172.21.15.92:0/3580027820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64b0004350 0x7f64b0004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 --2- 172.21.15.92:0/3580027820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b0004fd0 0x7f64b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc007a90 con 0x7f64b0004350 2024-04-22T07:04:13.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc0d5530 con 0x7f64b0004fd0 2024-04-22T07:04:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f64b000bfb0 con 0x7f64b00039b0 2024-04-22T07:04:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64ca540640 1 --2- 172.21.15.92:0/3580027820 >> v2:172.21.15.92:3300/0 conn(0x7f64bc004af0 0x7f64bc0d4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f9dbe953640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9db81577b0 0x7f9db8151730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9da800b3e0 tx=0x7f9da800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.923+0000 7f64cad41640 1 --2- 172.21.15.92:0/3580027820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b0004fd0 0x7f64b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.924+0000 7f64ca540640 1 --2- 172.21.15.92:0/3580027820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64b00039b0 0x7f64b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.924+0000 7f7637fff640 1 -- start start 2024-04-22T07:04:13.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.924+0000 7f441da38640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4418150b40 0x7f44180c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.924+0000 7f441da38640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44180c46b0 0x7f44180c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f441da38640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f441815aa60 con 0x7f44180c78c0 2024-04-22T07:04:13.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f441da38640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f44180c9960 con 0x7f44180c46b0 2024-04-22T07:04:13.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f441da38640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f44180c8410 con 0x7f4418150b40 2024-04-22T07:04:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f7637fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f76380c8f30 0x7f76380c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f90577fe640 1 --2- 172.21.15.92:0/1761929638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90580c63d0 0x7f905815f250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9050009d90 tx=0x7f905002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff09b29daf3ba8e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.924+0000 7f9daeffd640 1 -- 172.21.15.92:0/4240702392 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8015020 con 0x7f9db81577b0 2024-04-22T07:04:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.924+0000 7f9daeffd640 1 -- 172.21.15.92:0/4240702392 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da8011830 con 0x7f9db81577b0 2024-04-22T07:04:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.924+0000 7f9daeffd640 1 -- 172.21.15.92:0/4240702392 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8011ca0 con 0x7f9db81577b0 2024-04-22T07:04:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.924+0000 7f9dc03b6640 1 -- 172.21.15.92:0/4240702392 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db815ac30 con 0x7f9db81577b0 2024-04-22T07:04:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f441d237640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f44180c78c0 0x7f4418159f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46754/0 (socket says 172.21.15.92:46754) 2024-04-22T07:04:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f441d237640 1 -- 172.21.15.92:0/915660808 learned_addr learned my addr 172.21.15.92:0/915660808 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.924+0000 7f9dc03b6640 1 -- 172.21.15.92:0/4240702392 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db815af10 con 0x7f9db81577b0 2024-04-22T07:04:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f441ca36640 1 --2- 172.21.15.92:0/915660808 >> v2:172.21.15.145:3300/0 conn(0x7f4418150b40 0x7f44180c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f7637fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76380c6f50 0x7f76380c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f7637fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7638163310 con 0x7f76381591e0 2024-04-22T07:04:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f7637fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f763815f1a0 con 0x7f76380c6f50 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f9056ffd640 1 -- 172.21.15.92:0/1761929638 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050039070 con 0x7f90580c63d0 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f7637fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f76381595c0 con 0x7f76380c8f30 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f9056ffd640 1 -- 172.21.15.92:0/1761929638 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f905002cc80 con 0x7f90580c63d0 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f9056ffd640 1 -- 172.21.15.92:0/1761929638 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050034660 con 0x7f90580c63d0 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f76367fc640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f76380c8f30 0x7f76380c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f76367fc640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f76380c8f30 0x7f76380c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:54038/0 (socket says 172.21.15.92:54038) 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f76367fc640 1 -- 172.21.15.92:0/2753151719 learned_addr learned my addr 172.21.15.92:0/2753151719 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f76377fe640 1 --2- 172.21.15.92:0/2753151719 >> v2:172.21.15.92:3300/0 conn(0x7f76380c6f50 0x7f76380c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f441ca36640 1 -- 172.21.15.92:0/915660808 >> v2:172.21.15.92:3300/0 conn(0x7f44180c46b0 msgr2=0x7f44180c7380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f441ca36640 1 --2- 172.21.15.92:0/915660808 >> v2:172.21.15.92:3300/0 conn(0x7f44180c46b0 0x7f44180c7380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f441ca36640 1 -- 172.21.15.92:0/915660808 >> v1:172.21.15.92:6789/0 conn(0x7f44180c78c0 legacy=0x7f4418159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f441ca36640 1 -- 172.21.15.92:0/915660808 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441815ad20 con 0x7f4418150b40 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f44177fe640 1 -- 172.21.15.92:0/915660808 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3996495137 0 0) 0x7f441815aa60 con 0x7f44180c78c0 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f905e4e4640 1 -- 172.21.15.92:0/1761929638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90580c63d0 msgr2=0x7f905815f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f905e4e4640 1 --2- 172.21.15.92:0/1761929638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90580c63d0 0x7f905815f250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9050009d90 tx=0x7f905002bed0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f905e4e4640 1 -- 172.21.15.92:0/1761929638 shutdown_connections 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f905e4e4640 1 --2- 172.21.15.92:0/1761929638 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f905815f790 0x7f90580c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f905e4e4640 1 --2- 172.21.15.92:0/1761929638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90580c63d0 0x7f905815f250 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f905e4e4640 1 -- 172.21.15.92:0/1761929638 >> 172.21.15.92:0/1761929638 conn(0x7f90580cc220 msgr2=0x7f90580cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f905e4e4640 1 -- 172.21.15.92:0/1761929638 shutdown_connections 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.926+0000 7f9dc03b6640 1 -- 172.21.15.92:0/4240702392 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da0005290 con 0x7f9db81577b0 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f76377fe640 1 -- 172.21.15.92:0/2753151719 >> v2:172.21.15.145:3300/0 conn(0x7f76380c8f30 msgr2=0x7f76380c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f76377fe640 1 --2- 172.21.15.92:0/2753151719 >> v2:172.21.15.145:3300/0 conn(0x7f76380c8f30 0x7f76380c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f7635ffb640 1 -- 172.21.15.92:0/2753151719 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 88877475 0 0) 0x7f7638163310 con 0x7f76381591e0 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f76377fe640 1 -- 172.21.15.92:0/2753151719 >> v1:172.21.15.92:6789/0 conn(0x7f76381591e0 legacy=0x7f7638162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f76377fe640 1 -- 172.21.15.92:0/2753151719 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763815f460 con 0x7f76380c6f50 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f64ca540640 1 -- 172.21.15.92:0/3580027820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b0004fd0 msgr2=0x7f64b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f64ca540640 1 --2- 172.21.15.92:0/3580027820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b0004fd0 0x7f64b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f64ca540640 1 -- 172.21.15.92:0/3580027820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64b0004350 msgr2=0x7f64b0004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f64ca540640 1 --2- 172.21.15.92:0/3580027820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64b0004350 0x7f64b0004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.925+0000 7f64ca540640 1 -- 172.21.15.92:0/3580027820 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b0003520 con 0x7f64b00039b0 2024-04-22T07:04:13.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f441ca36640 1 --2- 172.21.15.92:0/915660808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4418150b40 0x7f44180c9420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44180c8590 tx=0x7f441002c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de6c4446323f4485 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.928+0000 7f44177fe640 1 -- 172.21.15.92:0/915660808 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441000a2d0 con 0x7f4418150b40 2024-04-22T07:04:13.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.928+0000 7f64ca540640 1 --2- 172.21.15.92:0/3580027820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64b00039b0 0x7f64b0003dd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f64c000aa80 tx=0x7f64b0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6f6ea3d3d40fdbd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.928+0000 7f44177fe640 1 -- 172.21.15.92:0/915660808 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f441000a470 con 0x7f4418150b40 2024-04-22T07:04:13.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.928+0000 7f44177fe640 1 -- 172.21.15.92:0/915660808 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441000a840 con 0x7f4418150b40 2024-04-22T07:04:13.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.928+0000 7f76377fe640 1 --2- 172.21.15.92:0/2753151719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76380c6f50 0x7f76380c73c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7638069970 tx=0x7f763002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb316dc4678e64f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f441da38640 1 -- 172.21.15.92:0/915660808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4418150b40 msgr2=0x7f44180c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f441da38640 1 --2- 172.21.15.92:0/915660808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4418150b40 0x7f44180c9420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44180c8590 tx=0x7f441002c350 comp rx=0 tx=0).stop 2024-04-22T07:04:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.928+0000 7f9daeffd640 1 -- 172.21.15.92:0/4240702392 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9da8018020 con 0x7f9db81577b0 2024-04-22T07:04:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.928+0000 7f9daeffd640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9d9c06c9c0 0x7f9d9c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.928+0000 7f9daeffd640 1 -- 172.21.15.92:0/4240702392 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9da808ba30 con 0x7f9db81577b0 2024-04-22T07:04:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.928+0000 7f0387967640 1 Processor -- start 2024-04-22T07:04:13.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.928+0000 7f0387967640 1 -- start start 2024-04-22T07:04:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f9dbe152640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9d9c06c9c0 0x7f9d9c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f7635ffb640 1 -- 172.21.15.92:0/2753151719 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630039070 con 0x7f76380c6f50 2024-04-22T07:04:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f441da38640 1 -- 172.21.15.92:0/915660808 shutdown_connections 2024-04-22T07:04:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f441da38640 1 --2- 172.21.15.92:0/915660808 >> v2:172.21.15.92:3300/0 conn(0x7f44180c46b0 0x7f44180c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f441da38640 1 --2- 172.21.15.92:0/915660808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4418150b40 0x7f44180c9420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f441da38640 1 -- 172.21.15.92:0/915660808 >> 172.21.15.92:0/915660808 conn(0x7f441806f8d0 msgr2=0x7f441806fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:13.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.927+0000 7f905e4e4640 1 -- 172.21.15.92:0/1761929638 wait complete. 2024-04-22T07:04:13.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f441da38640 1 -- 172.21.15.92:0/915660808 shutdown_connections 2024-04-22T07:04:13.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f7635ffb640 1 -- 172.21.15.92:0/2753151719 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f763002cbe0 con 0x7f76380c6f50 2024-04-22T07:04:13.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f7635ffb640 1 -- 172.21.15.92:0/2753151719 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630039070 con 0x7f76380c6f50 2024-04-22T07:04:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c000a650 con 0x7f64b00039b0 2024-04-22T07:04:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c002d630 con 0x7f64b00039b0 2024-04-22T07:04:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f64c953e640 1 -- 172.21.15.92:0/3580027820 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c002da80 con 0x7f64b00039b0 2024-04-22T07:04:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f0387967640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f03800c24d0 0x7f038015b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f0387967640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03800c8870 0x7f0380073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f0387967640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f038015a6b0 con 0x7f0380151450 2024-04-22T07:04:13.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f0387967640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f0380073b10 con 0x7f03800c24d0 2024-04-22T07:04:13.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f441da38640 1 -- 172.21.15.92:0/915660808 wait complete. 2024-04-22T07:04:13.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f0387967640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0380073dd0 con 0x7f03800c8870 2024-04-22T07:04:13.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f0385703640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0380151450 0x7f03800c9610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46774/0 (socket says 172.21.15.92:46774) 2024-04-22T07:04:13.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f0385703640 1 -- 172.21.15.92:0/767218372 learned_addr learned my addr 172.21.15.92:0/767218372 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:13.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f0385f04640 1 --2- 172.21.15.92:0/767218372 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03800c8870 0x7f0380073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f0384f02640 1 --2- 172.21.15.92:0/767218372 >> v2:172.21.15.145:3300/0 conn(0x7f03800c24d0 0x7f038015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f64cb542640 1 -- 172.21.15.92:0/3580027820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64b00039b0 msgr2=0x7f64b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f64cb542640 1 --2- 172.21.15.92:0/3580027820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64b00039b0 0x7f64b0003dd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f64c000aa80 tx=0x7f64b0005830 comp rx=0 tx=0).stop 2024-04-22T07:04:13.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f64cad41640 1 -- 172.21.15.92:0/3580027820 reap_dead start 2024-04-22T07:04:13.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.929+0000 7f905e4e4640 1 Processor -- start 2024-04-22T07:04:13.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f9dbe152640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9d9c06c9c0 0x7f9d9c06ee80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9db4036000 tx=0x7f9db4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.931+0000 7f7637fff640 1 -- 172.21.15.92:0/2753151719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76380c6f50 msgr2=0x7f76380c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.931+0000 7f7637fff640 1 --2- 172.21.15.92:0/2753151719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76380c6f50 0x7f76380c73c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7638069970 tx=0x7f763002be30 comp rx=0 tx=0).stop 2024-04-22T07:04:13.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.931+0000 7f441da38640 1 Processor -- start 2024-04-22T07:04:13.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.931+0000 7f441da38640 1 -- start start 2024-04-22T07:04:13.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f905e4e4640 1 -- start start 2024-04-22T07:04:13.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.931+0000 7f905e4e4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90580c63d0 0x7f90580cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.931+0000 7f905e4e4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90581592e0 0x7f905815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.931+0000 7f905e4e4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f905815f790 0x7f905815c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.931+0000 7f905e4e4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f905806f210 con 0x7f90580c63d0 2024-04-22T07:04:13.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f0377fff640 1 -- 172.21.15.92:0/767218372 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2927033312 0 0) 0x7f038015a6b0 con 0x7f0380151450 2024-04-22T07:04:13.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f0377fff640 1 -- 172.21.15.92:0/767218372 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f036c003520 con 0x7f0380151450 2024-04-22T07:04:13.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.931+0000 7f905e4e4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f90581596c0 con 0x7f90581592e0 2024-04-22T07:04:13.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.931+0000 7f905e4e4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f905806f390 con 0x7f905815f790 2024-04-22T07:04:13.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f0385f04640 1 -- 172.21.15.92:0/767218372 >> v2:172.21.15.145:3300/0 conn(0x7f03800c24d0 msgr2=0x7f038015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f0385f04640 1 --2- 172.21.15.92:0/767218372 >> v2:172.21.15.145:3300/0 conn(0x7f03800c24d0 0x7f038015b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f0385f04640 1 -- 172.21.15.92:0/767218372 >> v1:172.21.15.92:6789/0 conn(0x7f0380151450 legacy=0x7f03800c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.930+0000 7f0385f04640 1 -- 172.21.15.92:0/767218372 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038015a950 con 0x7f03800c8870 2024-04-22T07:04:13.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f0385f04640 1 --2- 172.21.15.92:0/767218372 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03800c8870 0x7f0380073460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f037800be30 tx=0x7f037800a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51e5a150adee2d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f7637fff640 1 -- 172.21.15.92:0/2753151719 shutdown_connections 2024-04-22T07:04:13.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f7637fff640 1 --2- 172.21.15.92:0/2753151719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76380c6f50 0x7f76380c73c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f7637fff640 1 --2- 172.21.15.92:0/2753151719 >> v2:172.21.15.145:3300/0 conn(0x7f76380c8f30 0x7f76380c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f7637fff640 1 -- 172.21.15.92:0/2753151719 >> 172.21.15.92:0/2753151719 conn(0x7f763806f8d0 msgr2=0x7f763806fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:13.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f90577fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90581592e0 0x7f905815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f905ca81640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f905815f790 0x7f905815c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f90577fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90581592e0 0x7f905815bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:54052/0 (socket says 172.21.15.92:54052) 2024-04-22T07:04:13.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f90577fe640 1 -- 172.21.15.92:0/632987159 learned_addr learned my addr 172.21.15.92:0/632987159 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:13.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f9057fff640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90580c63d0 0x7f90580cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f0377fff640 1 -- 172.21.15.92:0/767218372 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378032070 con 0x7f03800c8870 2024-04-22T07:04:13.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f0377fff640 1 -- 172.21.15.92:0/767218372 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f037802cd00 con 0x7f03800c8870 2024-04-22T07:04:13.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f0377fff640 1 -- 172.21.15.92:0/767218372 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378036620 con 0x7f03800c8870 2024-04-22T07:04:13.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f7637fff640 1 -- 172.21.15.92:0/2753151719 shutdown_connections 2024-04-22T07:04:13.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f7637fff640 1 -- 172.21.15.92:0/2753151719 wait complete. 2024-04-22T07:04:13.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f64cb542640 1 -- 172.21.15.92:0/3580027820 shutdown_connections 2024-04-22T07:04:13.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f64cb542640 1 -- 172.21.15.92:0/3580027820 >> 172.21.15.92:0/3580027820 conn(0x7f64bc0505c0 msgr2=0x7f64bc0509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f9057fff640 1 -- 172.21.15.92:0/632987159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f905815f790 msgr2=0x7f905815c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f9057fff640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f905815f790 0x7f905815c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f9057fff640 1 -- 172.21.15.92:0/632987159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90581592e0 msgr2=0x7f905815bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f9057fff640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90581592e0 0x7f905815bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f9057fff640 1 -- 172.21.15.92:0/632987159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905812d960 con 0x7f90580c63d0 2024-04-22T07:04:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f441da38640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44180c46b0 0x7f4418153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f441da38640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44180c78c0 0x7f4418200f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f441da38640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4418150b40 0x7f4418154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f441da38640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f44180c8c20 con 0x7f44180c46b0 2024-04-22T07:04:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f441da38640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f44180c8940 con 0x7f44180c78c0 2024-04-22T07:04:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.932+0000 7f441da38640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f44180c8f00 con 0x7f4418150b40 2024-04-22T07:04:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f64cb542640 1 -- 172.21.15.92:0/3580027820 shutdown_connections 2024-04-22T07:04:13.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f441d237640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4418150b40 0x7f4418154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f441ca36640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44180c46b0 0x7f4418153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f9057fff640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90580c63d0 0x7f90580cb930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c410 tx=0x7f904c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f441ca36640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44180c46b0 0x7f4418153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36478/0 (socket says 172.21.15.92:36478) 2024-04-22T07:04:13.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f441ca36640 1 -- 172.21.15.92:0/2731019272 learned_addr learned my addr 172.21.15.92:0/2731019272 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:13.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f0387967640 1 -- 172.21.15.92:0/767218372 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03800c8870 msgr2=0x7f0380073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f0387967640 1 --2- 172.21.15.92:0/767218372 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03800c8870 0x7f0380073460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f037800be30 tx=0x7f037800a400 comp rx=0 tx=0).stop 2024-04-22T07:04:13.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f9054ff9640 1 -- 172.21.15.92:0/632987159 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c017070 con 0x7f90580c63d0 2024-04-22T07:04:13.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.934+0000 7f441ca36640 1 -- 172.21.15.92:0/2731019272 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4418150b40 msgr2=0x7f4418154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.934+0000 7f441ca36640 1 --2- 172.21.15.92:0/2731019272 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4418150b40 0x7f4418154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.934+0000 7f441ca36640 1 -- 172.21.15.92:0/2731019272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44180c78c0 msgr2=0x7f4418200f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:13.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.934+0000 7f441ca36640 1 --2- 172.21.15.92:0/2731019272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44180c78c0 0x7f4418200f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.934+0000 7f441ca36640 1 -- 172.21.15.92:0/2731019272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44181ea920 con 0x7f44180c46b0 2024-04-22T07:04:13.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.934+0000 7f7637fff640 1 Processor -- start 2024-04-22T07:04:13.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.934+0000 7f441ca36640 1 --2- 172.21.15.92:0/2731019272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44180c46b0 0x7f4418153d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4410000f80 tx=0x7f4410004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f0387967640 1 -- 172.21.15.92:0/767218372 shutdown_connections 2024-04-22T07:04:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f0387967640 1 --2- 172.21.15.92:0/767218372 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03800c8870 0x7f0380073460 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f0387967640 1 --2- 172.21.15.92:0/767218372 >> v2:172.21.15.145:3300/0 conn(0x7f03800c24d0 0x7f038015b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f0387967640 1 -- 172.21.15.92:0/767218372 >> 172.21.15.92:0/767218372 conn(0x7f03800ccb80 msgr2=0x7f03800ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.934+0000 7f0387967640 1 -- 172.21.15.92:0/767218372 shutdown_connections 2024-04-22T07:04:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f64cb542640 1 -- 172.21.15.92:0/3580027820 wait complete. 2024-04-22T07:04:13.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.934+0000 7f64cb542640 1 Processor -- start 2024-04-22T07:04:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.934+0000 7f7637fff640 1 -- start start 2024-04-22T07:04:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f905e4e4640 1 -- 172.21.15.92:0/632987159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90581fdde0 con 0x7f90580c63d0 2024-04-22T07:04:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.933+0000 7f905e4e4640 1 -- 172.21.15.92:0/632987159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905815b420 con 0x7f90580c63d0 2024-04-22T07:04:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.934+0000 7f9054ff9640 1 -- 172.21.15.92:0/632987159 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904c005230 con 0x7f90580c63d0 2024-04-22T07:04:13.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.935+0000 7f9054ff9640 1 -- 172.21.15.92:0/632987159 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c0045e0 con 0x7f90580c63d0 2024-04-22T07:04:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.935+0000 7f7637fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76380c6f50 0x7f76381f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.935+0000 7f7637fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76380c8f30 0x7f763815bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.935+0000 7f7637fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76381591e0 0x7f76381542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.935+0000 7f7637fff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f763815fd00 con 0x7f76380c8f30 2024-04-22T07:04:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.935+0000 7f7637fff640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f763815fa20 con 0x7f76381591e0 2024-04-22T07:04:13.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.935+0000 7f64cb542640 1 -- start start 2024-04-22T07:04:13.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f64cb542640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64b00039b0 0x7f64bc049420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.935+0000 7f7637fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f76381626e0 con 0x7f76380c6f50 2024-04-22T07:04:13.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f9daeffd640 1 -- 172.21.15.92:0/4240702392 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9da801f020 con 0x7f9db81577b0 2024-04-22T07:04:13.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f7636ffd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76380c6f50 0x7f76381f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f64cb542640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64bc04f410 0x7f64bc035340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f64cb542640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64bc181fc0 0x7f64bc035fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.935+0000 7f0387967640 1 -- 172.21.15.92:0/767218372 wait complete. 2024-04-22T07:04:13.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f64ca540640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64b00039b0 0x7f64bc049420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f64ca540640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64b00039b0 0x7f64bc049420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36488/0 (socket says 172.21.15.92:36488) 2024-04-22T07:04:13.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f64ca540640 1 -- 172.21.15.92:0/186023360 learned_addr learned my addr 172.21.15.92:0/186023360 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:13.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc007720 con 0x7f64b00039b0 2024-04-22T07:04:13.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc007440 con 0x7f64bc04f410 2024-04-22T07:04:13.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f64bc04bb60 con 0x7f64bc181fc0 2024-04-22T07:04:13.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f64cad41640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64bc181fc0 0x7f64bc035fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f64c9d3f640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64bc04f410 0x7f64bc035340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f7636ffd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76380c6f50 0x7f76381f5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52676/0 (socket says 172.21.15.92:52676) 2024-04-22T07:04:13.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f7636ffd640 1 -- 172.21.15.92:0/3477937211 learned_addr learned my addr 172.21.15.92:0/3477937211 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:13.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f7636ffd640 1 -- 172.21.15.92:0/3477937211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76381591e0 msgr2=0x7f76381542a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:13.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f7636ffd640 1 --2- 172.21.15.92:0/3477937211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76381591e0 0x7f76381542a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.937+0000 7f905e4e4640 1 -- 172.21.15.92:0/632987159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9038005290 con 0x7f90580c63d0 2024-04-22T07:04:13.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f7636ffd640 1 -- 172.21.15.92:0/3477937211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76380c8f30 msgr2=0x7f763815bef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:04:13.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f7636ffd640 1 --2- 172.21.15.92:0/3477937211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76380c8f30 0x7f763815bef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.937+0000 7f64cad41640 1 -- 172.21.15.92:0/186023360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64bc04f410 msgr2=0x7f64bc035340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.937+0000 7f64cad41640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64bc04f410 0x7f64bc035340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.937+0000 7f64cad41640 1 -- 172.21.15.92:0/186023360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64b00039b0 msgr2=0x7f64bc049420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.937+0000 7f64cad41640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64b00039b0 0x7f64bc049420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.937+0000 7f64cad41640 1 -- 172.21.15.92:0/186023360 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc0348d0 con 0x7f64bc181fc0 2024-04-22T07:04:13.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.937+0000 7f64ca540640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64b00039b0 0x7f64bc049420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:13.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.937+0000 7f0387967640 1 Processor -- start 2024-04-22T07:04:13.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.937+0000 7f0387967640 1 -- start start 2024-04-22T07:04:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.937+0000 7f44157fa640 1 -- 172.21.15.92:0/2731019272 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410044020 con 0x7f44180c46b0 2024-04-22T07:04:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f441da38640 1 -- 172.21.15.92:0/2731019272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44181e95e0 con 0x7f44180c46b0 2024-04-22T07:04:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f441da38640 1 -- 172.21.15.92:0/2731019272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44181e9b60 con 0x7f44180c46b0 2024-04-22T07:04:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f0387967640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03800c24d0 0x7f038014c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f9054ff9640 1 -- 172.21.15.92:0/632987159 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f904c01a070 con 0x7f90580c63d0 2024-04-22T07:04:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f9054ff9640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f904406c9f0 0x7f904406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f44157fa640 1 -- 172.21.15.92:0/2731019272 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4410034d70 con 0x7f44180c46b0 2024-04-22T07:04:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f44157fa640 1 -- 172.21.15.92:0/2731019272 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44100406b0 con 0x7f44180c46b0 2024-04-22T07:04:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f0387967640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03800c8870 0x7f038014d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f0387967640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0380151450 0x7f038014d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f0387967640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03800c9ea0 con 0x7f0380151450 2024-04-22T07:04:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f9054ff9640 1 -- 172.21.15.92:0/632987159 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f904c08ad90 con 0x7f90580c63d0 2024-04-22T07:04:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f0387967640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f03800c9d20 con 0x7f03800c24d0 2024-04-22T07:04:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.938+0000 7f0387967640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0380157020 con 0x7f03800c8870 2024-04-22T07:04:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.936+0000 7f7636ffd640 1 -- 172.21.15.92:0/3477937211 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76381f55e0 con 0x7f76380c6f50 2024-04-22T07:04:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.939+0000 7f90577fe640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f904406c9f0 0x7f904406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.939+0000 7f7636ffd640 1 --2- 172.21.15.92:0/3477937211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76380c6f50 0x7f76381f5fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f762800c3f0 tx=0x7f762800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.939+0000 7f761f7fe640 1 -- 172.21.15.92:0/3477937211 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628017070 con 0x7f76380c6f50 2024-04-22T07:04:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.939+0000 7f761f7fe640 1 -- 172.21.15.92:0/3477937211 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76280035f0 con 0x7f76380c6f50 2024-04-22T07:04:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.939+0000 7f761f7fe640 1 -- 172.21.15.92:0/3477937211 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76280039e0 con 0x7f76380c6f50 2024-04-22T07:04:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.939+0000 7f90577fe640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f904406c9f0 0x7f904406eeb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f905815f410 tx=0x7f9050030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.939+0000 7f7637fff640 1 -- 172.21.15.92:0/3477937211 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76381f58c0 con 0x7f76380c6f50 2024-04-22T07:04:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.939+0000 7f7637fff640 1 -- 172.21.15.92:0/3477937211 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f76381f85e0 con 0x7f76380c6f50 2024-04-22T07:04:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.940+0000 7f0385f04640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0380151450 0x7f038014d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.940+0000 7f0385703640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03800c24d0 0x7f038014c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.940+0000 7f0385f04640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0380151450 0x7f038014d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36494/0 (socket says 172.21.15.92:36494) 2024-04-22T07:04:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.940+0000 7f0385f04640 1 -- 172.21.15.92:0/1364461569 learned_addr learned my addr 172.21.15.92:0/1364461569 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.940+0000 7f64cad41640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64bc181fc0 0x7f64bc035fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64c4074890 tx=0x7f64c4068ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.940+0000 7f0385f04640 1 -- 172.21.15.92:0/1364461569 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03800c8870 msgr2=0x7f038014d320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T07:04:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.940+0000 7f0385f04640 1 --2- 172.21.15.92:0/1364461569 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03800c8870 0x7f038014d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.940+0000 7f0385f04640 1 -- 172.21.15.92:0/1364461569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03800c24d0 msgr2=0x7f038014c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.940+0000 7f0385f04640 1 --2- 172.21.15.92:0/1364461569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03800c24d0 0x7f038014c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.940+0000 7f64baffd640 1 -- 172.21.15.92:0/186023360 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4093320 con 0x7f64bc181fc0 2024-04-22T07:04:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.940+0000 7f0385f04640 1 -- 172.21.15.92:0/1364461569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0380158380 con 0x7f0380151450 2024-04-22T07:04:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.940+0000 7f64baffd640 1 -- 172.21.15.92:0/186023360 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c4090020 con 0x7f64bc181fc0 2024-04-22T07:04:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.941+0000 7f0385f04640 1 --2- 172.21.15.92:0/1364461569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0380151450 0x7f038014d860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f037800bf50 tx=0x7f0378004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.941+0000 7f441da38640 1 -- 172.21.15.92:0/2731019272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f8005290 con 0x7f44180c46b0 2024-04-22T07:04:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.941+0000 7f7637fff640 1 -- 172.21.15.92:0/3477937211 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f763804fdc0 con 0x7f76380c6f50 2024-04-22T07:04:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.941+0000 7f64baffd640 1 -- 172.21.15.92:0/186023360 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4093720 con 0x7f64bc181fc0 2024-04-22T07:04:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.941+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64bc04fa60 con 0x7f64bc181fc0 2024-04-22T07:04:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.942+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64bc034b40 con 0x7f64bc181fc0 2024-04-22T07:04:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.942+0000 7f761f7fe640 1 -- 172.21.15.92:0/3477937211 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f762801a070 con 0x7f76380c6f50 2024-04-22T07:04:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.942+0000 7f761f7fe640 1 --2- 172.21.15.92:0/3477937211 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f762406c940 0x7f762406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.942+0000 7f761f7fe640 1 -- 172.21.15.92:0/3477937211 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f762808ade0 con 0x7f76380c6f50 2024-04-22T07:04:13.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.942+0000 7f76367fc640 1 --2- 172.21.15.92:0/3477937211 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f762406c940 0x7f762406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.943+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64bc04f7f0 con 0x7f64bc181fc0 2024-04-22T07:04:13.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.944+0000 7f76367fc640 1 --2- 172.21.15.92:0/3477937211 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f762406c940 0x7f762406ee00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f763815db80 tx=0x7f762c009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.944+0000 7f64baffd640 1 -- 172.21.15.92:0/186023360 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64c40938c0 con 0x7f64bc181fc0 2024-04-22T07:04:13.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.944+0000 7f64baffd640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f64a806c9c0 0x7f64a806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.944+0000 7f64baffd640 1 -- 172.21.15.92:0/186023360 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f64c4107160 con 0x7f64bc181fc0 2024-04-22T07:04:13.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.944+0000 7f64ca540640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f64a806c9c0 0x7f64a806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.945+0000 7f64ca540640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f64a806c9c0 0x7f64a806ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f64c000a1a0 tx=0x7f64c0031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.947+0000 7f761f7fe640 1 -- 172.21.15.92:0/3477937211 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7628059380 con 0x7f76380c6f50 2024-04-22T07:04:13.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.948+0000 7f9054ff9640 1 -- 172.21.15.92:0/632987159 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f904c058be0 con 0x7f90580c63d0 2024-04-22T07:04:13.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.950+0000 7f44157fa640 1 -- 172.21.15.92:0/2731019272 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4410047020 con 0x7f44180c46b0 2024-04-22T07:04:13.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.951+0000 7f44157fa640 1 --2- 172.21.15.92:0/2731019272 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f440406c9c0 0x7f440406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.951+0000 7f4417fff640 1 --2- 172.21.15.92:0/2731019272 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f440406c9c0 0x7f440406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.951+0000 7f44157fa640 1 -- 172.21.15.92:0/2731019272 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f44100af530 con 0x7f44180c46b0 2024-04-22T07:04:13.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.951+0000 7f64baffd640 1 -- 172.21.15.92:0/186023360 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64c409a030 con 0x7f64bc181fc0 2024-04-22T07:04:13.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.955+0000 7f4417fff640 1 --2- 172.21.15.92:0/2731019272 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f440406c9c0 0x7f440406ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f440c006fd0 tx=0x7f440c006b40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.958+0000 7f0375ffb640 1 -- 172.21.15.92:0/1364461569 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378045020 con 0x7f0380151450 2024-04-22T07:04:13.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.958+0000 7f0387967640 1 -- 172.21.15.92:0/1364461569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0380158660 con 0x7f0380151450 2024-04-22T07:04:13.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.958+0000 7f0387967640 1 -- 172.21.15.92:0/1364461569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0380158c20 con 0x7f0380151450 2024-04-22T07:04:13.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.960+0000 7f44157fa640 1 -- 172.21.15.92:0/2731019272 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f441004e030 con 0x7f44180c46b0 2024-04-22T07:04:13.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.959+0000 7f0375ffb640 1 -- 172.21.15.92:0/1364461569 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f037800ad50 con 0x7f0380151450 2024-04-22T07:04:13.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.960+0000 7f0375ffb640 1 -- 172.21.15.92:0/1364461569 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03780416b0 con 0x7f0380151450 2024-04-22T07:04:13.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.963+0000 7f0375ffb640 1 -- 172.21.15.92:0/1364461569 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0378048020 con 0x7f0380151450 2024-04-22T07:04:13.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.963+0000 7f0375ffb640 1 --2- 172.21.15.92:0/1364461569 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f036406c9c0 0x7f036406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.964+0000 7f0385703640 1 --2- 172.21.15.92:0/1364461569 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f036406c9c0 0x7f036406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.964+0000 7f0385703640 1 --2- 172.21.15.92:0/1364461569 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f036406c9c0 0x7f036406ee80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f037c005880 tx=0x7f037c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.965+0000 7f0375ffb640 1 -- 172.21.15.92:0/1364461569 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f03780ae3d0 con 0x7f0380151450 2024-04-22T07:04:13.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.965+0000 7f0387967640 1 -- 172.21.15.92:0/1364461569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0360005290 con 0x7f0380151450 2024-04-22T07:04:13.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:13.974+0000 7f0375ffb640 1 -- 172.21.15.92:0/1364461569 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f037807c750 con 0x7f0380151450 2024-04-22T07:04:14.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.235+0000 7f991cb38640 1 -- 172.21.15.92:0/3782591514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f98fc005580 con 0x7f99180b9550 2024-04-22T07:04:14.236 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2024-04-22T07:04:14.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.235+0000 7f9906ffd640 1 -- 172.21.15.92:0/3782591514 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f990c059740 con 0x7f99180b9550 2024-04-22T07:04:14.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.241+0000 7f9904ff9640 1 -- 172.21.15.92:0/3782591514 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f98f806c9c0 msgr2=0x7f98f806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.241+0000 7f9904ff9640 1 --2- 172.21.15.92:0/3782591514 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f98f806c9c0 0x7f98f806ee80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99080014f0 tx=0x7f9908007990 comp rx=0 tx=0).stop 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.241+0000 7f9904ff9640 1 -- 172.21.15.92:0/3782591514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99180b9550 msgr2=0x7f99180c5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.241+0000 7f9904ff9640 1 --2- 172.21.15.92:0/3782591514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99180b9550 0x7f99180c5520 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f990c00b890 tx=0x7f990c00bc90 comp rx=0 tx=0).stop 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.241+0000 7f9904ff9640 1 -- 172.21.15.92:0/3782591514 shutdown_connections 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.241+0000 7f9904ff9640 1 --2- 172.21.15.92:0/3782591514 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f98f806c9c0 0x7f98f806ee80 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99080014f0 tx=0x7f9908007990 comp rx=0 tx=0).stop 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.241+0000 7f9904ff9640 1 --2- 172.21.15.92:0/3782591514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99180b9550 0x7f99180c5520 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.241+0000 7f9904ff9640 1 --2- 172.21.15.92:0/3782591514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f991806f5d0 0x7f991815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.241+0000 7f9904ff9640 1 --2- 172.21.15.92:0/3782591514 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f991806eb00 0x7f9918158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.241+0000 7f9904ff9640 1 -- 172.21.15.92:0/3782591514 >> 172.21.15.92:0/3782591514 conn(0x7f99180cc220 msgr2=0x7f9918162990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.243+0000 7f9904ff9640 1 -- 172.21.15.92:0/3782591514 shutdown_connections 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.243+0000 7f9904ff9640 1 -- 172.21.15.92:0/3782591514 wait complete. 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.242+0000 7f4dce5fa640 1 -- 172.21.15.92:0/2055101119 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4dc0006960 con 0x7f4dc00d9250 2024-04-22T07:04:14.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.253+0000 7f4dbdffb640 1 -- 172.21.15.92:0/2055101119 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4dc80d5ea0 con 0x7f4dc00d9250 2024-04-22T07:04:14.253 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:14.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-04-22T07:04:14.259 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.261+0000 7f4dce5fa640 1 -- 172.21.15.92:0/2055101119 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4db406ca40 msgr2=0x7f4db406ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.261+0000 7f4dce5fa640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4db406ca40 0x7f4db406ef00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009560 tx=0x7f4dc4009190 comp rx=0 tx=0).stop 2024-04-22T07:04:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.261+0000 7f4dce5fa640 1 -- 172.21.15.92:0/2055101119 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 msgr2=0x7f4dc0038fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.261+0000 7f4dce5fa640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 0x7f4dc0038fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4dc806b920 tx=0x7f4dc80695e0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.262+0000 7f4dce5fa640 1 -- 172.21.15.92:0/2055101119 shutdown_connections 2024-04-22T07:04:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.262+0000 7f4dce5fa640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4db406ca40 0x7f4db406ef00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.262+0000 7f4dce5fa640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc00e0490 0x7f4dc0039500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.262+0000 7f4dce5fa640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4dc00d9250 0x7f4dc0038fc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.262+0000 7f4dce5fa640 1 --2- 172.21.15.92:0/2055101119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4dc004ffd0 0x7f4dc00df0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.262+0000 7f4dce5fa640 1 -- 172.21.15.92:0/2055101119 >> 172.21.15.92:0/2055101119 conn(0x7f4dc000e160 msgr2=0x7f4dc00d6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.262+0000 7f4dce5fa640 1 -- 172.21.15.92:0/2055101119 shutdown_connections 2024-04-22T07:04:14.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.262+0000 7f4dce5fa640 1 -- 172.21.15.92:0/2055101119 wait complete. 2024-04-22T07:04:14.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-04-22T07:04:14.276 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:14.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.371+0000 7f905e4e4640 1 -- 172.21.15.92:0/632987159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9038005580 con 0x7f90580c63d0 2024-04-22T07:04:14.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.371+0000 7f9054ff9640 1 -- 172.21.15.92:0/632987159 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f904c05cee0 con 0x7f90580c63d0 2024-04-22T07:04:14.370 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2024-04-22T07:04:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.374+0000 7f905e4e4640 1 -- 172.21.15.92:0/632987159 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f904406c9f0 msgr2=0x7f904406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.374+0000 7f905e4e4640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f904406c9f0 0x7f904406eeb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f905815f410 tx=0x7f9050030000 comp rx=0 tx=0).stop 2024-04-22T07:04:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.374+0000 7f905e4e4640 1 -- 172.21.15.92:0/632987159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90580c63d0 msgr2=0x7f90580cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.374+0000 7f905e4e4640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90580c63d0 0x7f90580cb930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c410 tx=0x7f904c00c8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.375+0000 7f905e4e4640 1 -- 172.21.15.92:0/632987159 shutdown_connections 2024-04-22T07:04:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.375+0000 7f905e4e4640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f904406c9f0 0x7f904406eeb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.375+0000 7f905e4e4640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f905815f790 0x7f905815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.375+0000 7f905e4e4640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90581592e0 0x7f905815bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.375+0000 7f905e4e4640 1 --2- 172.21.15.92:0/632987159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90580c63d0 0x7f90580cb930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.375+0000 7f905e4e4640 1 -- 172.21.15.92:0/632987159 >> 172.21.15.92:0/632987159 conn(0x7f90580cc220 msgr2=0x7f905814e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.375+0000 7f905e4e4640 1 -- 172.21.15.92:0/632987159 shutdown_connections 2024-04-22T07:04:14.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.375+0000 7f905e4e4640 1 -- 172.21.15.92:0/632987159 wait complete. 2024-04-22T07:04:14.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.376+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f64bc0057d0 con 0x7f64bc181fc0 2024-04-22T07:04:14.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.376+0000 7f64baffd640 1 -- 172.21.15.92:0/186023360 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f64c40d5570 con 0x7f64bc181fc0 2024-04-22T07:04:14.374 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2024-04-22T07:04:14.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.380+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f64a806c9c0 msgr2=0x7f64a806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.380+0000 7f64cb542640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f64a806c9c0 0x7f64a806ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f64c000a1a0 tx=0x7f64c0031000 comp rx=0 tx=0).stop 2024-04-22T07:04:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.380+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64bc181fc0 msgr2=0x7f64bc035fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.380+0000 7f64cb542640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64bc181fc0 0x7f64bc035fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64c4074890 tx=0x7f64c4068ed0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.380+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 shutdown_connections 2024-04-22T07:04:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.380+0000 7f64cb542640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f64a806c9c0 0x7f64a806ee80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.380+0000 7f64cb542640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64bc181fc0 0x7f64bc035fd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.380+0000 7f64cb542640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64bc04f410 0x7f64bc035340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.380+0000 7f64cb542640 1 --2- 172.21.15.92:0/186023360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64b00039b0 0x7f64bc049420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.380+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 >> 172.21.15.92:0/186023360 conn(0x7f64bc0505c0 msgr2=0x7f64bc0059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.380+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 shutdown_connections 2024-04-22T07:04:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.380+0000 7f64cb542640 1 -- 172.21.15.92:0/186023360 wait complete. 2024-04-22T07:04:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.384+0000 7f441da38640 1 -- 172.21.15.92:0/2731019272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f43f8005850 con 0x7f44180c46b0 2024-04-22T07:04:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.384+0000 7f44157fa640 1 -- 172.21.15.92:0/2731019272 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f441007dbb0 con 0x7f44180c46b0 2024-04-22T07:04:14.382 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:14.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.388+0000 7f441da38640 1 -- 172.21.15.92:0/2731019272 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f440406c9c0 msgr2=0x7f440406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.388+0000 7f441da38640 1 --2- 172.21.15.92:0/2731019272 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f440406c9c0 0x7f440406ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f440c006fd0 tx=0x7f440c006b40 comp rx=0 tx=0).stop 2024-04-22T07:04:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.388+0000 7f441da38640 1 -- 172.21.15.92:0/2731019272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44180c46b0 msgr2=0x7f4418153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.388+0000 7f441da38640 1 --2- 172.21.15.92:0/2731019272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44180c46b0 0x7f4418153d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4410000f80 tx=0x7f4410004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.388+0000 7f441da38640 1 -- 172.21.15.92:0/2731019272 shutdown_connections 2024-04-22T07:04:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.388+0000 7f441da38640 1 --2- 172.21.15.92:0/2731019272 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f440406c9c0 0x7f440406ee80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.388+0000 7f441da38640 1 --2- 172.21.15.92:0/2731019272 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4418150b40 0x7f4418154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.388+0000 7f441da38640 1 --2- 172.21.15.92:0/2731019272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44180c78c0 0x7f4418200f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.388+0000 7f441da38640 1 --2- 172.21.15.92:0/2731019272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44180c46b0 0x7f4418153d60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.388+0000 7f441da38640 1 -- 172.21.15.92:0/2731019272 >> 172.21.15.92:0/2731019272 conn(0x7f441806f8d0 msgr2=0x7f44180c4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.388+0000 7f441da38640 1 -- 172.21.15.92:0/2731019272 shutdown_connections 2024-04-22T07:04:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.388+0000 7f441da38640 1 -- 172.21.15.92:0/2731019272 wait complete. 2024-04-22T07:04:14.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-04-22T07:04:14.390 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:14.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-04-22T07:04:14.394 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:14.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-04-22T07:04:14.402 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:14.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.414+0000 7f7637fff640 1 -- 172.21.15.92:0/3477937211 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f76380c7ab0 con 0x7f76380c6f50 2024-04-22T07:04:14.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.415+0000 7f761f7fe640 1 -- 172.21.15.92:0/3477937211 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7628058c80 con 0x7f76380c6f50 2024-04-22T07:04:14.413 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:14.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.417+0000 7f7637fff640 1 -- 172.21.15.92:0/3477937211 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f762406c940 msgr2=0x7f762406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.417+0000 7f7637fff640 1 --2- 172.21.15.92:0/3477937211 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f762406c940 0x7f762406ee00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f763815db80 tx=0x7f762c009240 comp rx=0 tx=0).stop 2024-04-22T07:04:14.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.417+0000 7f7637fff640 1 -- 172.21.15.92:0/3477937211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76380c6f50 msgr2=0x7f76381f5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.417+0000 7f7637fff640 1 --2- 172.21.15.92:0/3477937211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76380c6f50 0x7f76381f5fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f762800c3f0 tx=0x7f762800c8b0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.417+0000 7f7637fff640 1 -- 172.21.15.92:0/3477937211 shutdown_connections 2024-04-22T07:04:14.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.417+0000 7f7637fff640 1 --2- 172.21.15.92:0/3477937211 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f762406c940 0x7f762406ee00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.417+0000 7f7637fff640 1 --2- 172.21.15.92:0/3477937211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76381591e0 0x7f76381542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.417+0000 7f7637fff640 1 --2- 172.21.15.92:0/3477937211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76380c8f30 0x7f763815bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.417+0000 7f7637fff640 1 --2- 172.21.15.92:0/3477937211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76380c6f50 0x7f76381f5fb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.417+0000 7f7637fff640 1 -- 172.21.15.92:0/3477937211 >> 172.21.15.92:0/3477937211 conn(0x7f763806f8d0 msgr2=0x7f7638062390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.417+0000 7f7637fff640 1 -- 172.21.15.92:0/3477937211 shutdown_connections 2024-04-22T07:04:14.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.417+0000 7f7637fff640 1 -- 172.21.15.92:0/3477937211 wait complete. 2024-04-22T07:04:14.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-04-22T07:04:14.426 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:14.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.456+0000 7f0387967640 1 -- 172.21.15.92:0/1364461569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0360005580 con 0x7f0380151450 2024-04-22T07:04:14.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.456+0000 7f0375ffb640 1 -- 172.21.15.92:0/1364461569 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f037807c050 con 0x7f0380151450 2024-04-22T07:04:14.454 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2024-04-22T07:04:14.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.459+0000 7f0387967640 1 -- 172.21.15.92:0/1364461569 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f036406c9c0 msgr2=0x7f036406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.459+0000 7f0387967640 1 --2- 172.21.15.92:0/1364461569 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f036406c9c0 0x7f036406ee80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f037c005880 tx=0x7f037c009040 comp rx=0 tx=0).stop 2024-04-22T07:04:14.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.459+0000 7f0387967640 1 -- 172.21.15.92:0/1364461569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0380151450 msgr2=0x7f038014d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.459+0000 7f0387967640 1 --2- 172.21.15.92:0/1364461569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0380151450 0x7f038014d860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f037800bf50 tx=0x7f0378004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:14.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.459+0000 7f0387967640 1 -- 172.21.15.92:0/1364461569 shutdown_connections 2024-04-22T07:04:14.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.459+0000 7f0387967640 1 --2- 172.21.15.92:0/1364461569 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f036406c9c0 0x7f036406ee80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.459+0000 7f0387967640 1 --2- 172.21.15.92:0/1364461569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0380151450 0x7f038014d860 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.459+0000 7f0387967640 1 --2- 172.21.15.92:0/1364461569 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03800c8870 0x7f038014d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.459+0000 7f0387967640 1 --2- 172.21.15.92:0/1364461569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03800c24d0 0x7f038014c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.459+0000 7f0387967640 1 -- 172.21.15.92:0/1364461569 >> 172.21.15.92:0/1364461569 conn(0x7f03800ccb80 msgr2=0x7f03800c9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.459+0000 7f0387967640 1 -- 172.21.15.92:0/1364461569 shutdown_connections 2024-04-22T07:04:14.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.459+0000 7f0387967640 1 -- 172.21.15.92:0/1364461569 wait complete. 2024-04-22T07:04:14.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.460+0000 7f9dc03b6640 1 -- 172.21.15.92:0/4240702392 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9da0005580 con 0x7f9db81577b0 2024-04-22T07:04:14.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.461+0000 7f9daeffd640 1 -- 172.21.15.92:0/4240702392 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9da8059db0 con 0x7f9db81577b0 2024-04-22T07:04:14.459 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2024-04-22T07:04:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.463+0000 7f9dc03b6640 1 -- 172.21.15.92:0/4240702392 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9d9c06c9c0 msgr2=0x7f9d9c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.463+0000 7f9dc03b6640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9d9c06c9c0 0x7f9d9c06ee80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9db4036000 tx=0x7f9db4008030 comp rx=0 tx=0).stop 2024-04-22T07:04:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.463+0000 7f9dc03b6640 1 -- 172.21.15.92:0/4240702392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9db81577b0 msgr2=0x7f9db8151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.463+0000 7f9dc03b6640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9db81577b0 0x7f9db8151730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9da800b3e0 tx=0x7f9da800b8a0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.463+0000 7f9dc03b6640 1 -- 172.21.15.92:0/4240702392 shutdown_connections 2024-04-22T07:04:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.463+0000 7f9dc03b6640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9d9c06c9c0 0x7f9d9c06ee80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.463+0000 7f9dc03b6640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9db81577b0 0x7f9db8151730 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.463+0000 7f9dc03b6640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9db81510b0 0x7f9db81593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.463+0000 7f9dc03b6640 1 --2- 172.21.15.92:0/4240702392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9db806ffb0 0x7f9db81f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.463+0000 7f9dc03b6640 1 -- 172.21.15.92:0/4240702392 >> 172.21.15.92:0/4240702392 conn(0x7f9db80cc220 msgr2=0x7f9db814f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.463+0000 7f9dc03b6640 1 -- 172.21.15.92:0/4240702392 shutdown_connections 2024-04-22T07:04:14.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.463+0000 7f9dc03b6640 1 -- 172.21.15.92:0/4240702392 wait complete. 2024-04-22T07:04:14.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-04-22T07:04:14.469 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:14.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-22T07:04:14.472 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:14.472 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-22T07:04:14.473 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:14.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.612+0000 7fc53faf1640 1 Processor -- start 2024-04-22T07:04:14.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.612+0000 7fc53faf1640 1 -- start start 2024-04-22T07:04:14.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53faf1640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc5380c43c0 0x7fc538070490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53faf1640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5380720e0 con 0x7fc538156190 2024-04-22T07:04:14.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53faf1640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc538072350 con 0x7fc5380c8df0 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53faf1640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc5380c3410 con 0x7fc5380c43c0 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53d88d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc538156190 0x7fc5380c3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46776/0 (socket says 172.21.15.92:46776) 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53d88d640 1 -- 172.21.15.92:0/811567336 learned_addr learned my addr 172.21.15.92:0/811567336 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53d08c640 1 --2- 172.21.15.92:0/811567336 >> v2:172.21.15.145:3300/0 conn(0x7fc5380c43c0 0x7fc538070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 133976239 0 0) 0x7fc5380720e0 con 0x7fc538156190 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc524003520 con 0x7fc538156190 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1872376713 0 0) 0x7fc524003520 con 0x7fc538156190 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 >> v2:172.21.15.145:3300/0 conn(0x7fc5380c43c0 msgr2=0x7fc538070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53c88b640 1 --2- 172.21.15.92:0/811567336 >> v2:172.21.15.145:3300/0 conn(0x7fc5380c43c0 0x7fc538070490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 >> v1:172.21.15.145:6789/0 conn(0x7fc5380c8df0 legacy=0x7fc538163330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5380c3660 con 0x7fc538156190 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7fc534001690 con 0x7fc538156190 2024-04-22T07:04:14.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 >> v1:172.21.15.92:6789/0 conn(0x7fc538156190 legacy=0x7fc5380c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53c88b640 1 --2- 172.21.15.92:0/811567336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5240039b0 0x7fc524003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.613+0000 7fc53c88b640 1 --2- 172.21.15.92:0/811567336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc524004800 0x7fc524004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53c88b640 1 --2- 172.21.15.92:0/811567336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5240051c0 0x7fc52400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc5380c3410 con 0x7fc5240039b0 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc52400c090 con 0x7fc5240051c0 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc52400c290 con 0x7fc524004800 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53d88d640 1 --2- 172.21.15.92:0/811567336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5240039b0 0x7fc524003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53e08e640 1 --2- 172.21.15.92:0/811567336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5240051c0 0x7fc52400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53d08c640 1 --2- 172.21.15.92:0/811567336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc524004800 0x7fc524004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53d88d640 1 -- 172.21.15.92:0/811567336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc524004800 msgr2=0x7fc524004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53d88d640 1 --2- 172.21.15.92:0/811567336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc524004800 0x7fc524004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53d88d640 1 -- 172.21.15.92:0/811567336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5240051c0 msgr2=0x7fc52400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53d88d640 1 --2- 172.21.15.92:0/811567336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5240051c0 0x7fc52400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53e08e640 1 -- 172.21.15.92:0/811567336 reap_dead start 2024-04-22T07:04:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.614+0000 7fc53d88d640 1 -- 172.21.15.92:0/811567336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc524003520 con 0x7fc5240039b0 2024-04-22T07:04:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.615+0000 7fc53d88d640 1 --2- 172.21.15.92:0/811567336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5240039b0 0x7fc524003df0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c210 tx=0x7fc524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37aa5af243f36e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.615+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534007d10 con 0x7fc5240039b0 2024-04-22T07:04:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.615+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc534030070 con 0x7fc5240039b0 2024-04-22T07:04:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.615+0000 7fc53c88b640 1 -- 172.21.15.92:0/811567336 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534005d30 con 0x7fc5240039b0 2024-04-22T07:04:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.615+0000 7fc53faf1640 1 -- 172.21.15.92:0/811567336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5240039b0 msgr2=0x7fc524003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.615+0000 7fc53faf1640 1 --2- 172.21.15.92:0/811567336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5240039b0 0x7fc524003df0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c210 tx=0x7fc524005830 comp rx=0 tx=0).stop 2024-04-22T07:04:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.615+0000 7fc53faf1640 1 -- 172.21.15.92:0/811567336 shutdown_connections 2024-04-22T07:04:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.615+0000 7fc53faf1640 1 --2- 172.21.15.92:0/811567336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5240039b0 0x7fc524003df0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.615+0000 7fc53faf1640 1 -- 172.21.15.92:0/811567336 >> 172.21.15.92:0/811567336 conn(0x7fc53806e260 msgr2=0x7fc5380715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.615+0000 7fc53faf1640 1 -- 172.21.15.92:0/811567336 shutdown_connections 2024-04-22T07:04:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.615+0000 7fc53faf1640 1 -- 172.21.15.92:0/811567336 wait complete. 2024-04-22T07:04:14.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.616+0000 7fc53faf1640 1 Processor -- start 2024-04-22T07:04:14.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.617+0000 7fc53faf1640 1 -- start start 2024-04-22T07:04:14.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.617+0000 7fc53faf1640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5240039b0 0x7fc5380ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.617+0000 7fc53faf1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380cc820 0x7fc5381e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.617+0000 7fc53faf1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5380cedd0 0x7fc5381e4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.617+0000 7fc53faf1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc53806f9b0 con 0x7fc5380cc820 2024-04-22T07:04:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.617+0000 7fc53faf1640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc538070000 con 0x7fc5380cedd0 2024-04-22T07:04:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.617+0000 7fc53faf1640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc53806fc90 con 0x7fc5240039b0 2024-04-22T07:04:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.617+0000 7fc53d08c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380cc820 0x7fc5381e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.617+0000 7fc53d08c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380cc820 0x7fc5381e3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36504/0 (socket says 172.21.15.92:36504) 2024-04-22T07:04:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.617+0000 7fc53d08c640 1 -- 172.21.15.92:0/1338350856 learned_addr learned my addr 172.21.15.92:0/1338350856 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.617+0000 7fc53d88d640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5240039b0 0x7fc5380ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.617+0000 7fc53e08e640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5380cedd0 0x7fc5381e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.618+0000 7fc53d08c640 1 -- 172.21.15.92:0/1338350856 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5240039b0 msgr2=0x7fc5380ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.618+0000 7fc53d08c640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5240039b0 0x7fc5380ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.618+0000 7fc53d08c640 1 -- 172.21.15.92:0/1338350856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5380cedd0 msgr2=0x7fc5381e4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.618+0000 7fc53d08c640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5380cedd0 0x7fc5381e4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.618+0000 7fc53d08c640 1 -- 172.21.15.92:0/1338350856 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5381f12f0 con 0x7fc5380cc820 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.618+0000 7fc53d08c640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380cc820 0x7fc5381e3cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc528011ad0 tx=0x7fc528011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.618+0000 7fc52e7fc640 1 -- 172.21.15.92:0/1338350856 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52800fbb0 con 0x7fc5380cc820 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.618+0000 7fc52e7fc640 1 -- 172.21.15.92:0/1338350856 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc52800fd50 con 0x7fc5380cc820 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.618+0000 7fc52e7fc640 1 -- 172.21.15.92:0/1338350856 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528014660 con 0x7fc5380cc820 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.618+0000 7fc53faf1640 1 -- 172.21.15.92:0/1338350856 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5381f1590 con 0x7fc5380cc820 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.618+0000 7fc53faf1640 1 -- 172.21.15.92:0/1338350856 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5381e31e0 con 0x7fc5380cc820 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.619+0000 7fc53faf1640 1 -- 172.21.15.92:0/1338350856 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc538068b00 con 0x7fc5380cc820 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.619+0000 7fc52e7fc640 1 -- 172.21.15.92:0/1338350856 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc528012040 con 0x7fc5380cc820 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.620+0000 7fc52e7fc640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc51c06c920 0x7fc51c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.620+0000 7fc52e7fc640 1 -- 172.21.15.92:0/1338350856 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc52808bfd0 con 0x7fc5380cc820 2024-04-22T07:04:14.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.620+0000 7fc53d88d640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc51c06c920 0x7fc51c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.620+0000 7fc53d88d640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc51c06c920 0x7fc51c06ede0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c210 tx=0x7fc534007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:14.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.624+0000 7fc52e7fc640 1 -- 172.21.15.92:0/1338350856 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc528019030 con 0x7fc5380cc820 2024-04-22T07:04:14.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.800+0000 7fc53faf1640 1 -- 172.21.15.92:0/1338350856 --> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5380b43e0 con 0x7fc51c06c920 2024-04-22T07:04:14.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.806+0000 7fc52e7fc640 1 -- 172.21.15.92:0/1338350856 <== mgr.4107 v2:172.21.15.92:6832/479317737 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fc5340827d0 con 0x7fc51c06c920 2024-04-22T07:04:14.807 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-22T07:04:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.809+0000 7fc53faf1640 1 -- 172.21.15.92:0/1338350856 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc51c06c920 msgr2=0x7fc51c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.809+0000 7fc53faf1640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc51c06c920 0x7fc51c06ede0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c210 tx=0x7fc534007000 comp rx=0 tx=0).stop 2024-04-22T07:04:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.809+0000 7fc53faf1640 1 -- 172.21.15.92:0/1338350856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380cc820 msgr2=0x7fc5381e3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.809+0000 7fc53faf1640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380cc820 0x7fc5381e3cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc528011ad0 tx=0x7fc528011f90 comp rx=0 tx=0).stop 2024-04-22T07:04:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.810+0000 7fc53faf1640 1 -- 172.21.15.92:0/1338350856 shutdown_connections 2024-04-22T07:04:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.810+0000 7fc53faf1640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc51c06c920 0x7fc51c06ede0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.810+0000 7fc53faf1640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5380cedd0 0x7fc5381e4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.810+0000 7fc53faf1640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380cc820 0x7fc5381e3cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.810+0000 7fc53faf1640 1 --2- 172.21.15.92:0/1338350856 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5240039b0 0x7fc5380ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.810+0000 7fc53faf1640 1 -- 172.21.15.92:0/1338350856 >> 172.21.15.92:0/1338350856 conn(0x7fc53806e260 msgr2=0x7fc5380b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.810+0000 7fc53faf1640 1 -- 172.21.15.92:0/1338350856 shutdown_connections 2024-04-22T07:04:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.810+0000 7fc53faf1640 1 -- 172.21.15.92:0/1338350856 wait complete. 2024-04-22T07:04:14.818 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-22T07:04:13.934904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218952,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755768,"statfs":{"total":773094113280,"available":772869906432,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001905"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862171+0000","last_change":"2024-04-22T07:04:03.796924+0000","last_active":"2024-04-22T07:04:04.862171+0000","last_peered":"2024-04-22T07:04:04.862171+0000","last_clean":"2024-04-22T07:04:04.862171+0000","last_became_active":"2024-04-22T07:04:03.796652+0000","last_became_peered":"2024-04-22T07:04:03.796652+0000","last_unstale":"2024-04-22T07:04:04.862171+0000","last_undegraded":"2024-04-22T07:04:04.862171+0000","last_fullsized":"2024-04-22T07:04:04.862171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:01.867906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788117+0000","last_change":"2024-04-22T07:04:03.793652+0000","last_active":"2024-04-22T07:04:04.788117+0000","last_peered":"2024-04-22T07:04:04.788117+0000","last_clean":"2024-04-22T07:04:04.788117+0000","last_became_active":"2024-04-22T07:04:03.793397+0000","last_became_peered":"2024-04-22T07:04:03.793397+0000","last_unstale":"2024-04-22T07:04:04.788117+0000","last_undegraded":"2024-04-22T07:04:04.788117+0000","last_fullsized":"2024-04-22T07:04:04.788117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859908+0000","last_change":"2024-04-22T07:04:03.788534+0000","last_active":"2024-04-22T07:04:04.859908+0000","last_peered":"2024-04-22T07:04:04.859908+0000","last_clean":"2024-04-22T07:04:04.859908+0000","last_became_active":"2024-04-22T07:04:03.787747+0000","last_became_peered":"2024-04-22T07:04:03.787747+0000","last_unstale":"2024-04-22T07:04:04.859908+0000","last_undegraded":"2024-04-22T07:04:04.859908+0000","last_fullsized":"2024-04-22T07:04:04.859908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861337+0000","last_change":"2024-04-22T07:04:03.800150+0000","last_active":"2024-04-22T07:04:04.861337+0000","last_peered":"2024-04-22T07:04:04.861337+0000","last_clean":"2024-04-22T07:04:04.861337+0000","last_became_active":"2024-04-22T07:04:03.799887+0000","last_became_peered":"2024-04-22T07:04:03.799887+0000","last_unstale":"2024-04-22T07:04:04.861337+0000","last_undegraded":"2024-04-22T07:04:04.861337+0000","last_fullsized":"2024-04-22T07:04:04.861337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861727+0000","last_change":"2024-04-22T07:04:03.798482+0000","last_active":"2024-04-22T07:04:04.861727+0000","last_peered":"2024-04-22T07:04:04.861727+0000","last_clean":"2024-04-22T07:04:04.861727+0000","last_became_active":"2024-04-22T07:04:03.798205+0000","last_became_peered":"2024-04-22T07:04:03.798205+0000","last_unstale":"2024-04-22T07:04:04.861727+0000","last_undegraded":"2024-04-22T07:04:04.861727+0000","last_fullsized":"2024-04-22T07:04:04.861727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861044+0000","last_change":"2024-04-22T07:04:03.797373+0000","last_active":"2024-04-22T07:04:04.861044+0000","last_peered":"2024-04-22T07:04:04.861044+0000","last_clean":"2024-04-22T07:04:04.861044+0000","last_became_active":"2024-04-22T07:04:03.797090+0000","last_became_peered":"2024-04-22T07:04:03.797090+0000","last_unstale":"2024-04-22T07:04:04.861044+0000","last_undegraded":"2024-04-22T07:04:04.861044+0000","last_fullsized":"2024-04-22T07:04:04.861044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861437+0000","last_change":"2024-04-22T07:04:03.793725+0000","last_active":"2024-04-22T07:04:04.861437+0000","last_peered":"2024-04-22T07:04:04.861437+0000","last_clean":"2024-04-22T07:04:04.861437+0000","last_became_active":"2024-04-22T07:04:03.793418+0000","last_became_peered":"2024-04-22T07:04:03.793418+0000","last_unstale":"2024-04-22T07:04:04.861437+0000","last_undegraded":"2024-04-22T07:04:04.861437+0000","last_fullsized":"2024-04-22T07:04:04.861437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859500+0000","last_change":"2024-04-22T07:04:03.795377+0000","last_active":"2024-04-22T07:04:04.859500+0000","last_peered":"2024-04-22T07:04:04.859500+0000","last_clean":"2024-04-22T07:04:04.859500+0000","last_became_active":"2024-04-22T07:04:03.795106+0000","last_became_peered":"2024-04-22T07:04:03.795106+0000","last_unstale":"2024-04-22T07:04:04.859500+0000","last_undegraded":"2024-04-22T07:04:04.859500+0000","last_fullsized":"2024-04-22T07:04:04.859500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862209+0000","last_change":"2024-04-22T07:04:03.797692+0000","last_active":"2024-04-22T07:04:04.862209+0000","last_peered":"2024-04-22T07:04:04.862209+0000","last_clean":"2024-04-22T07:04:04.862209+0000","last_became_active":"2024-04-22T07:04:03.797396+0000","last_became_peered":"2024-04-22T07:04:03.797396+0000","last_unstale":"2024-04-22T07:04:04.862209+0000","last_undegraded":"2024-04-22T07:04:04.862209+0000","last_fullsized":"2024-04-22T07:04:04.862209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:39.068857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860742+0000","last_change":"2024-04-22T07:04:03.797228+0000","last_active":"2024-04-22T07:04:04.860742+0000","last_peered":"2024-04-22T07:04:04.860742+0000","last_clean":"2024-04-22T07:04:04.860742+0000","last_became_active":"2024-04-22T07:04:03.796614+0000","last_became_peered":"2024-04-22T07:04:03.796614+0000","last_unstale":"2024-04-22T07:04:04.860742+0000","last_undegraded":"2024-04-22T07:04:04.860742+0000","last_fullsized":"2024-04-22T07:04:04.860742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860497+0000","last_change":"2024-04-22T07:04:03.799434+0000","last_active":"2024-04-22T07:04:04.860497+0000","last_peered":"2024-04-22T07:04:04.860497+0000","last_clean":"2024-04-22T07:04:04.860497+0000","last_became_active":"2024-04-22T07:04:03.799173+0000","last_became_peered":"2024-04-22T07:04:03.799173+0000","last_unstale":"2024-04-22T07:04:04.860497+0000","last_undegraded":"2024-04-22T07:04:04.860497+0000","last_fullsized":"2024-04-22T07:04:04.860497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790488+0000","last_change":"2024-04-22T07:04:03.800035+0000","last_active":"2024-04-22T07:04:04.790488+0000","last_peered":"2024-04-22T07:04:04.790488+0000","last_clean":"2024-04-22T07:04:04.790488+0000","last_became_active":"2024-04-22T07:04:03.799646+0000","last_became_peered":"2024-04-22T07:04:03.799646+0000","last_unstale":"2024-04-22T07:04:04.790488+0000","last_undegraded":"2024-04-22T07:04:04.790488+0000","last_fullsized":"2024-04-22T07:04:04.790488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:17.287068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860651+0000","last_change":"2024-04-22T07:04:03.797324+0000","last_active":"2024-04-22T07:04:04.860651+0000","last_peered":"2024-04-22T07:04:04.860651+0000","last_clean":"2024-04-22T07:04:04.860651+0000","last_became_active":"2024-04-22T07:04:03.797039+0000","last_became_peered":"2024-04-22T07:04:03.797039+0000","last_unstale":"2024-04-22T07:04:04.860651+0000","last_undegraded":"2024-04-22T07:04:04.860651+0000","last_fullsized":"2024-04-22T07:04:04.860651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859720+0000","last_change":"2024-04-22T07:04:03.790732+0000","last_active":"2024-04-22T07:04:04.859720+0000","last_peered":"2024-04-22T07:04:04.859720+0000","last_clean":"2024-04-22T07:04:04.859720+0000","last_became_active":"2024-04-22T07:04:03.788340+0000","last_became_peered":"2024-04-22T07:04:03.788340+0000","last_unstale":"2024-04-22T07:04:04.859720+0000","last_undegraded":"2024-04-22T07:04:04.859720+0000","last_fullsized":"2024-04-22T07:04:04.859720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787860+0000","last_change":"2024-04-22T07:04:03.799368+0000","last_active":"2024-04-22T07:04:04.787860+0000","last_peered":"2024-04-22T07:04:04.787860+0000","last_clean":"2024-04-22T07:04:04.787860+0000","last_became_active":"2024-04-22T07:04:03.799067+0000","last_became_peered":"2024-04-22T07:04:03.799067+0000","last_unstale":"2024-04-22T07:04:04.787860+0000","last_undegraded":"2024-04-22T07:04:04.787860+0000","last_fullsized":"2024-04-22T07:04:04.787860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860988+0000","last_change":"2024-04-22T07:04:03.802947+0000","last_active":"2024-04-22T07:04:04.860988+0000","last_peered":"2024-04-22T07:04:04.860988+0000","last_clean":"2024-04-22T07:04:04.860988+0000","last_became_active":"2024-04-22T07:04:03.802493+0000","last_became_peered":"2024-04-22T07:04:03.802493+0000","last_unstale":"2024-04-22T07:04:04.860988+0000","last_undegraded":"2024-04-22T07:04:04.860988+0000","last_fullsized":"2024-04-22T07:04:04.860988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860714+0000","last_change":"2024-04-22T07:04:03.795581+0000","last_active":"2024-04-22T07:04:04.860714+0000","last_peered":"2024-04-22T07:04:04.860714+0000","last_clean":"2024-04-22T07:04:04.860714+0000","last_became_active":"2024-04-22T07:04:03.795292+0000","last_became_peered":"2024-04-22T07:04:03.795292+0000","last_unstale":"2024-04-22T07:04:04.860714+0000","last_undegraded":"2024-04-22T07:04:04.860714+0000","last_fullsized":"2024-04-22T07:04:04.860714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859993+0000","last_change":"2024-04-22T07:04:03.796132+0000","last_active":"2024-04-22T07:04:04.859993+0000","last_peered":"2024-04-22T07:04:04.859993+0000","last_clean":"2024-04-22T07:04:04.859993+0000","last_became_active":"2024-04-22T07:04:03.795069+0000","last_became_peered":"2024-04-22T07:04:03.795069+0000","last_unstale":"2024-04-22T07:04:04.859993+0000","last_undegraded":"2024-04-22T07:04:04.859993+0000","last_fullsized":"2024-04-22T07:04:04.859993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:08.934970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862125+0000","last_change":"2024-04-22T07:04:03.794552+0000","last_active":"2024-04-22T07:04:04.862125+0000","last_peered":"2024-04-22T07:04:04.862125+0000","last_clean":"2024-04-22T07:04:04.862125+0000","last_became_active":"2024-04-22T07:04:03.794046+0000","last_became_peered":"2024-04-22T07:04:03.794046+0000","last_unstale":"2024-04-22T07:04:04.862125+0000","last_undegraded":"2024-04-22T07:04:04.862125+0000","last_fullsized":"2024-04-22T07:04:04.862125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860703+0000","last_change":"2024-04-22T07:04:03.798686+0000","last_active":"2024-04-22T07:04:04.860703+0000","last_peered":"2024-04-22T07:04:04.860703+0000","last_clean":"2024-04-22T07:04:04.860703+0000","last_became_active":"2024-04-22T07:04:03.798104+0000","last_became_peered":"2024-04-22T07:04:03.798104+0000","last_unstale":"2024-04-22T07:04:04.860703+0000","last_undegraded":"2024-04-22T07:04:04.860703+0000","last_fullsized":"2024-04-22T07:04:04.860703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860494+0000","last_change":"2024-04-22T07:04:03.794215+0000","last_active":"2024-04-22T07:04:04.860494+0000","last_peered":"2024-04-22T07:04:04.860494+0000","last_clean":"2024-04-22T07:04:04.860494+0000","last_became_active":"2024-04-22T07:04:03.793990+0000","last_became_peered":"2024-04-22T07:04:03.793990+0000","last_unstale":"2024-04-22T07:04:04.860494+0000","last_undegraded":"2024-04-22T07:04:04.860494+0000","last_fullsized":"2024-04-22T07:04:04.860494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861162+0000","last_change":"2024-04-22T07:04:03.798539+0000","last_active":"2024-04-22T07:04:04.861162+0000","last_peered":"2024-04-22T07:04:04.861162+0000","last_clean":"2024-04-22T07:04:04.861162+0000","last_became_active":"2024-04-22T07:04:03.797713+0000","last_became_peered":"2024-04-22T07:04:03.797713+0000","last_unstale":"2024-04-22T07:04:04.861162+0000","last_undegraded":"2024-04-22T07:04:04.861162+0000","last_fullsized":"2024-04-22T07:04:04.861162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859945+0000","last_change":"2024-04-22T07:04:03.798610+0000","last_active":"2024-04-22T07:04:04.859945+0000","last_peered":"2024-04-22T07:04:04.859945+0000","last_clean":"2024-04-22T07:04:04.859945+0000","last_became_active":"2024-04-22T07:04:03.797981+0000","last_became_peered":"2024-04-22T07:04:03.797981+0000","last_unstale":"2024-04-22T07:04:04.859945+0000","last_undegraded":"2024-04-22T07:04:04.859945+0000","last_fullsized":"2024-04-22T07:04:04.859945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861507+0000","last_change":"2024-04-22T07:04:03.802742+0000","last_active":"2024-04-22T07:04:04.861507+0000","last_peered":"2024-04-22T07:04:04.861507+0000","last_clean":"2024-04-22T07:04:04.861507+0000","last_became_active":"2024-04-22T07:04:03.802127+0000","last_became_peered":"2024-04-22T07:04:03.802127+0000","last_unstale":"2024-04-22T07:04:04.861507+0000","last_undegraded":"2024-04-22T07:04:04.861507+0000","last_fullsized":"2024-04-22T07:04:04.861507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860138+0000","last_change":"2024-04-22T07:04:03.798789+0000","last_active":"2024-04-22T07:04:04.860138+0000","last_peered":"2024-04-22T07:04:04.860138+0000","last_clean":"2024-04-22T07:04:04.860138+0000","last_became_active":"2024-04-22T07:04:03.798505+0000","last_became_peered":"2024-04-22T07:04:03.798505+0000","last_unstale":"2024-04-22T07:04:04.860138+0000","last_undegraded":"2024-04-22T07:04:04.860138+0000","last_fullsized":"2024-04-22T07:04:04.860138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860919+0000","last_change":"2024-04-22T07:04:03.794111+0000","last_active":"2024-04-22T07:04:04.860919+0000","last_peered":"2024-04-22T07:04:04.860919+0000","last_clean":"2024-04-22T07:04:04.860919+0000","last_became_active":"2024-04-22T07:04:03.793682+0000","last_became_peered":"2024-04-22T07:04:03.793682+0000","last_unstale":"2024-04-22T07:04:04.860919+0000","last_undegraded":"2024-04-22T07:04:04.860919+0000","last_fullsized":"2024-04-22T07:04:04.860919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788083+0000","last_change":"2024-04-22T07:04:03.790167+0000","last_active":"2024-04-22T07:04:04.788083+0000","last_peered":"2024-04-22T07:04:04.788083+0000","last_clean":"2024-04-22T07:04:04.788083+0000","last_became_active":"2024-04-22T07:04:03.789873+0000","last_became_peered":"2024-04-22T07:04:03.789873+0000","last_unstale":"2024-04-22T07:04:04.788083+0000","last_undegraded":"2024-04-22T07:04:04.788083+0000","last_fullsized":"2024-04-22T07:04:04.788083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859954+0000","last_change":"2024-04-22T07:04:03.798713+0000","last_active":"2024-04-22T07:04:04.859954+0000","last_peered":"2024-04-22T07:04:04.859954+0000","last_clean":"2024-04-22T07:04:04.859954+0000","last_became_active":"2024-04-22T07:04:03.798344+0000","last_became_peered":"2024-04-22T07:04:03.798344+0000","last_unstale":"2024-04-22T07:04:04.859954+0000","last_undegraded":"2024-04-22T07:04:04.859954+0000","last_fullsized":"2024-04-22T07:04:04.859954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860634+0000","last_change":"2024-04-22T07:04:03.795452+0000","last_active":"2024-04-22T07:04:04.860634+0000","last_peered":"2024-04-22T07:04:04.860634+0000","last_clean":"2024-04-22T07:04:04.860634+0000","last_became_active":"2024-04-22T07:04:03.794970+0000","last_became_peered":"2024-04-22T07:04:03.794970+0000","last_unstale":"2024-04-22T07:04:04.860634+0000","last_undegraded":"2024-04-22T07:04:04.860634+0000","last_fullsized":"2024-04-22T07:04:04.860634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861203+0000","last_change":"2024-04-22T07:04:03.799581+0000","last_active":"2024-04-22T07:04:04.861203+0000","last_peered":"2024-04-22T07:04:04.861203+0000","last_clean":"2024-04-22T07:04:04.861203+0000","last_became_active":"2024-04-22T07:04:03.799328+0000","last_became_peered":"2024-04-22T07:04:03.799328+0000","last_unstale":"2024-04-22T07:04:04.861203+0000","last_undegraded":"2024-04-22T07:04:04.861203+0000","last_fullsized":"2024-04-22T07:04:04.861203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859944+0000","last_change":"2024-04-22T07:04:03.796830+0000","last_active":"2024-04-22T07:04:04.859944+0000","last_peered":"2024-04-22T07:04:04.859944+0000","last_clean":"2024-04-22T07:04:04.859944+0000","last_became_active":"2024-04-22T07:04:03.796551+0000","last_became_peered":"2024-04-22T07:04:03.796551+0000","last_unstale":"2024-04-22T07:04:04.859944+0000","last_undegraded":"2024-04-22T07:04:04.859944+0000","last_fullsized":"2024-04-22T07:04:04.859944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860101+0000","last_change":"2024-04-22T07:04:03.795320+0000","last_active":"2024-04-22T07:04:04.860101+0000","last_peered":"2024-04-22T07:04:04.860101+0000","last_clean":"2024-04-22T07:04:04.860101+0000","last_became_active":"2024-04-22T07:04:03.795025+0000","last_became_peered":"2024-04-22T07:04:03.795025+0000","last_unstale":"2024-04-22T07:04:04.860101+0000","last_undegraded":"2024-04-22T07:04:04.860101+0000","last_fullsized":"2024-04-22T07:04:04.860101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790119+0000","last_change":"2024-04-22T07:04:03.788859+0000","last_active":"2024-04-22T07:04:04.790119+0000","last_peered":"2024-04-22T07:04:04.790119+0000","last_clean":"2024-04-22T07:04:04.790119+0000","last_became_active":"2024-04-22T07:04:03.788573+0000","last_became_peered":"2024-04-22T07:04:03.788573+0000","last_unstale":"2024-04-22T07:04:04.790119+0000","last_undegraded":"2024-04-22T07:04:04.790119+0000","last_fullsized":"2024-04-22T07:04:04.790119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:36.716689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860730+0000","last_change":"2024-04-22T07:04:02.795724+0000","last_active":"2024-04-22T07:04:04.860730+0000","last_peered":"2024-04-22T07:04:04.860730+0000","last_clean":"2024-04-22T07:04:04.860730+0000","last_became_active":"2024-04-22T07:04:02.795371+0000","last_became_peered":"2024-04-22T07:04:02.795371+0000","last_unstale":"2024-04-22T07:04:04.860730+0000","last_undegraded":"2024-04-22T07:04:04.860730+0000","last_fullsized":"2024-04-22T07:04:04.860730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860689+0000","last_change":"2024-04-22T07:04:03.787030+0000","last_active":"2024-04-22T07:04:04.860689+0000","last_peered":"2024-04-22T07:04:04.860689+0000","last_clean":"2024-04-22T07:04:04.860689+0000","last_became_active":"2024-04-22T07:04:03.786357+0000","last_became_peered":"2024-04-22T07:04:03.786357+0000","last_unstale":"2024-04-22T07:04:04.860689+0000","last_undegraded":"2024-04-22T07:04:04.860689+0000","last_fullsized":"2024-04-22T07:04:04.860689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860368+0000","last_change":"2024-04-22T07:04:02.791429+0000","last_active":"2024-04-22T07:04:04.860368+0000","last_peered":"2024-04-22T07:04:04.860368+0000","last_clean":"2024-04-22T07:04:04.860368+0000","last_became_active":"2024-04-22T07:04:02.791114+0000","last_became_peered":"2024-04-22T07:04:02.791114+0000","last_unstale":"2024-04-22T07:04:04.860368+0000","last_undegraded":"2024-04-22T07:04:04.860368+0000","last_fullsized":"2024-04-22T07:04:04.860368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860040+0000","last_change":"2024-04-22T07:04:02.789531+0000","last_active":"2024-04-22T07:04:04.860040+0000","last_peered":"2024-04-22T07:04:04.860040+0000","last_clean":"2024-04-22T07:04:04.860040+0000","last_became_active":"2024-04-22T07:04:02.789248+0000","last_became_peered":"2024-04-22T07:04:02.789248+0000","last_unstale":"2024-04-22T07:04:04.860040+0000","last_undegraded":"2024-04-22T07:04:04.860040+0000","last_fullsized":"2024-04-22T07:04:04.860040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:06:43.031782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859624+0000","last_change":"2024-04-22T07:04:03.790142+0000","last_active":"2024-04-22T07:04:04.859624+0000","last_peered":"2024-04-22T07:04:04.859624+0000","last_clean":"2024-04-22T07:04:04.859624+0000","last_became_active":"2024-04-22T07:04:03.789869+0000","last_became_peered":"2024-04-22T07:04:03.789869+0000","last_unstale":"2024-04-22T07:04:04.859624+0000","last_undegraded":"2024-04-22T07:04:04.859624+0000","last_fullsized":"2024-04-22T07:04:04.859624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861050+0000","last_change":"2024-04-22T07:04:02.790537+0000","last_active":"2024-04-22T07:04:04.861050+0000","last_peered":"2024-04-22T07:04:04.861050+0000","last_clean":"2024-04-22T07:04:04.861050+0000","last_became_active":"2024-04-22T07:04:02.789072+0000","last_became_peered":"2024-04-22T07:04:02.789072+0000","last_unstale":"2024-04-22T07:04:04.861050+0000","last_undegraded":"2024-04-22T07:04:04.861050+0000","last_fullsized":"2024-04-22T07:04:04.861050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787835+0000","last_change":"2024-04-22T07:04:03.791239+0000","last_active":"2024-04-22T07:04:04.787835+0000","last_peered":"2024-04-22T07:04:04.787835+0000","last_clean":"2024-04-22T07:04:04.787835+0000","last_became_active":"2024-04-22T07:04:03.790515+0000","last_became_peered":"2024-04-22T07:04:03.790515+0000","last_unstale":"2024-04-22T07:04:04.787835+0000","last_undegraded":"2024-04-22T07:04:04.787835+0000","last_fullsized":"2024-04-22T07:04:04.787835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860666+0000","last_change":"2024-04-22T07:04:02.793582+0000","last_active":"2024-04-22T07:04:04.860666+0000","last_peered":"2024-04-22T07:04:04.860666+0000","last_clean":"2024-04-22T07:04:04.860666+0000","last_became_active":"2024-04-22T07:04:02.793281+0000","last_became_peered":"2024-04-22T07:04:02.793281+0000","last_unstale":"2024-04-22T07:04:04.860666+0000","last_undegraded":"2024-04-22T07:04:04.860666+0000","last_fullsized":"2024-04-22T07:04:04.860666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860510+0000","last_change":"2024-04-22T07:04:03.793994+0000","last_active":"2024-04-22T07:04:04.860510+0000","last_peered":"2024-04-22T07:04:04.860510+0000","last_clean":"2024-04-22T07:04:04.860510+0000","last_became_active":"2024-04-22T07:04:03.793734+0000","last_became_peered":"2024-04-22T07:04:03.793734+0000","last_unstale":"2024-04-22T07:04:04.860510+0000","last_undegraded":"2024-04-22T07:04:04.860510+0000","last_fullsized":"2024-04-22T07:04:04.860510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860572+0000","last_change":"2024-04-22T07:04:02.791945+0000","last_active":"2024-04-22T07:04:04.860572+0000","last_peered":"2024-04-22T07:04:04.860572+0000","last_clean":"2024-04-22T07:04:04.860572+0000","last_became_active":"2024-04-22T07:04:02.791654+0000","last_became_peered":"2024-04-22T07:04:02.791654+0000","last_unstale":"2024-04-22T07:04:04.860572+0000","last_undegraded":"2024-04-22T07:04:04.860572+0000","last_fullsized":"2024-04-22T07:04:04.860572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860230+0000","last_change":"2024-04-22T07:04:03.797391+0000","last_active":"2024-04-22T07:04:04.860230+0000","last_peered":"2024-04-22T07:04:04.860230+0000","last_clean":"2024-04-22T07:04:04.860230+0000","last_became_active":"2024-04-22T07:04:03.797123+0000","last_became_peered":"2024-04-22T07:04:03.797123+0000","last_unstale":"2024-04-22T07:04:04.860230+0000","last_undegraded":"2024-04-22T07:04:04.860230+0000","last_fullsized":"2024-04-22T07:04:04.860230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.785515+0000","last_change":"2024-04-22T07:04:02.793753+0000","last_active":"2024-04-22T07:04:08.785515+0000","last_peered":"2024-04-22T07:04:08.785515+0000","last_clean":"2024-04-22T07:04:08.785515+0000","last_became_active":"2024-04-22T07:04:02.793436+0000","last_became_peered":"2024-04-22T07:04:02.793436+0000","last_unstale":"2024-04-22T07:04:08.785515+0000","last_undegraded":"2024-04-22T07:04:08.785515+0000","last_fullsized":"2024-04-22T07:04:08.785515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790527+0000","last_change":"2024-04-22T07:04:03.800665+0000","last_active":"2024-04-22T07:04:04.790527+0000","last_peered":"2024-04-22T07:04:04.790527+0000","last_clean":"2024-04-22T07:04:04.790527+0000","last_became_active":"2024-04-22T07:04:03.800227+0000","last_became_peered":"2024-04-22T07:04:03.800227+0000","last_unstale":"2024-04-22T07:04:04.790527+0000","last_undegraded":"2024-04-22T07:04:04.790527+0000","last_fullsized":"2024-04-22T07:04:04.790527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.784027+0000","last_change":"2024-04-22T07:04:02.791309+0000","last_active":"2024-04-22T07:04:08.784027+0000","last_peered":"2024-04-22T07:04:08.784027+0000","last_clean":"2024-04-22T07:04:08.784027+0000","last_became_active":"2024-04-22T07:04:02.791000+0000","last_became_peered":"2024-04-22T07:04:02.791000+0000","last_unstale":"2024-04-22T07:04:08.784027+0000","last_undegraded":"2024-04-22T07:04:08.784027+0000","last_fullsized":"2024-04-22T07:04:08.784027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861543+0000","last_change":"2024-04-22T07:04:03.794316+0000","last_active":"2024-04-22T07:04:04.861543+0000","last_peered":"2024-04-22T07:04:04.861543+0000","last_clean":"2024-04-22T07:04:04.861543+0000","last_became_active":"2024-04-22T07:04:03.793646+0000","last_became_peered":"2024-04-22T07:04:03.793646+0000","last_unstale":"2024-04-22T07:04:04.861543+0000","last_undegraded":"2024-04-22T07:04:04.861543+0000","last_fullsized":"2024-04-22T07:04:04.861543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.782279+0000","last_change":"2024-04-22T07:04:02.791024+0000","last_active":"2024-04-22T07:04:08.782279+0000","last_peered":"2024-04-22T07:04:08.782279+0000","last_clean":"2024-04-22T07:04:08.782279+0000","last_became_active":"2024-04-22T07:04:02.789160+0000","last_became_peered":"2024-04-22T07:04:02.789160+0000","last_unstale":"2024-04-22T07:04:08.782279+0000","last_undegraded":"2024-04-22T07:04:08.782279+0000","last_fullsized":"2024-04-22T07:04:08.782279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859792+0000","last_change":"2024-04-22T07:04:03.798436+0000","last_active":"2024-04-22T07:04:04.859792+0000","last_peered":"2024-04-22T07:04:04.859792+0000","last_clean":"2024-04-22T07:04:04.859792+0000","last_became_active":"2024-04-22T07:04:03.797946+0000","last_became_peered":"2024-04-22T07:04:03.797946+0000","last_unstale":"2024-04-22T07:04:04.859792+0000","last_undegraded":"2024-04-22T07:04:04.859792+0000","last_fullsized":"2024-04-22T07:04:04.859792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861234+0000","last_change":"2024-04-22T07:04:02.796551+0000","last_active":"2024-04-22T07:04:04.861234+0000","last_peered":"2024-04-22T07:04:04.861234+0000","last_clean":"2024-04-22T07:04:04.861234+0000","last_became_active":"2024-04-22T07:04:02.796332+0000","last_became_peered":"2024-04-22T07:04:02.796332+0000","last_unstale":"2024-04-22T07:04:04.861234+0000","last_undegraded":"2024-04-22T07:04:04.861234+0000","last_fullsized":"2024-04-22T07:04:04.861234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:15.854295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861758+0000","last_change":"2024-04-22T07:04:03.804622+0000","last_active":"2024-04-22T07:04:04.861758+0000","last_peered":"2024-04-22T07:04:04.861758+0000","last_clean":"2024-04-22T07:04:04.861758+0000","last_became_active":"2024-04-22T07:04:03.804324+0000","last_became_peered":"2024-04-22T07:04:03.804324+0000","last_unstale":"2024-04-22T07:04:04.861758+0000","last_undegraded":"2024-04-22T07:04:04.861758+0000","last_fullsized":"2024-04-22T07:04:04.861758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860167+0000","last_change":"2024-04-22T07:04:02.787085+0000","last_active":"2024-04-22T07:04:04.860167+0000","last_peered":"2024-04-22T07:04:04.860167+0000","last_clean":"2024-04-22T07:04:04.860167+0000","last_became_active":"2024-04-22T07:04:02.786777+0000","last_became_peered":"2024-04-22T07:04:02.786777+0000","last_unstale":"2024-04-22T07:04:04.860167+0000","last_undegraded":"2024-04-22T07:04:04.860167+0000","last_fullsized":"2024-04-22T07:04:04.860167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860535+0000","last_change":"2024-04-22T07:04:03.796356+0000","last_active":"2024-04-22T07:04:04.860535+0000","last_peered":"2024-04-22T07:04:04.860535+0000","last_clean":"2024-04-22T07:04:04.860535+0000","last_became_active":"2024-04-22T07:04:03.796030+0000","last_became_peered":"2024-04-22T07:04:03.796030+0000","last_unstale":"2024-04-22T07:04:04.860535+0000","last_undegraded":"2024-04-22T07:04:04.860535+0000","last_fullsized":"2024-04-22T07:04:04.860535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.786454+0000","last_change":"2024-04-22T07:04:02.787425+0000","last_active":"2024-04-22T07:04:08.786454+0000","last_peered":"2024-04-22T07:04:08.786454+0000","last_clean":"2024-04-22T07:04:08.786454+0000","last_became_active":"2024-04-22T07:04:02.787146+0000","last_became_peered":"2024-04-22T07:04:02.787146+0000","last_unstale":"2024-04-22T07:04:08.786454+0000","last_undegraded":"2024-04-22T07:04:08.786454+0000","last_fullsized":"2024-04-22T07:04:08.786454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861631+0000","last_change":"2024-04-22T07:04:03.802924+0000","last_active":"2024-04-22T07:04:04.861631+0000","last_peered":"2024-04-22T07:04:04.861631+0000","last_clean":"2024-04-22T07:04:04.861631+0000","last_became_active":"2024-04-22T07:04:03.802644+0000","last_became_peered":"2024-04-22T07:04:03.802644+0000","last_unstale":"2024-04-22T07:04:04.861631+0000","last_undegraded":"2024-04-22T07:04:04.861631+0000","last_fullsized":"2024-04-22T07:04:04.861631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790421+0000","last_change":"2024-04-22T07:04:02.794481+0000","last_active":"2024-04-22T07:04:04.790421+0000","last_peered":"2024-04-22T07:04:04.790421+0000","last_clean":"2024-04-22T07:04:04.790421+0000","last_became_active":"2024-04-22T07:04:02.793982+0000","last_became_peered":"2024-04-22T07:04:02.793982+0000","last_unstale":"2024-04-22T07:04:04.790421+0000","last_undegraded":"2024-04-22T07:04:04.790421+0000","last_fullsized":"2024-04-22T07:04:04.790421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:58.906416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862139+0000","last_change":"2024-04-22T07:04:03.802724+0000","last_active":"2024-04-22T07:04:04.862139+0000","last_peered":"2024-04-22T07:04:04.862139+0000","last_clean":"2024-04-22T07:04:04.862139+0000","last_became_active":"2024-04-22T07:04:03.802434+0000","last_became_peered":"2024-04-22T07:04:03.802434+0000","last_unstale":"2024-04-22T07:04:04.862139+0000","last_undegraded":"2024-04-22T07:04:04.862139+0000","last_fullsized":"2024-04-22T07:04:04.862139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.781655+0000","last_change":"2024-04-22T07:04:02.791375+0000","last_active":"2024-04-22T07:04:08.781655+0000","last_peered":"2024-04-22T07:04:08.781655+0000","last_clean":"2024-04-22T07:04:08.781655+0000","last_became_active":"2024-04-22T07:04:02.791095+0000","last_became_peered":"2024-04-22T07:04:02.791095+0000","last_unstale":"2024-04-22T07:04:08.781655+0000","last_undegraded":"2024-04-22T07:04:08.781655+0000","last_fullsized":"2024-04-22T07:04:08.781655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862265+0000","last_change":"2024-04-22T07:04:03.802028+0000","last_active":"2024-04-22T07:04:04.862265+0000","last_peered":"2024-04-22T07:04:04.862265+0000","last_clean":"2024-04-22T07:04:04.862265+0000","last_became_active":"2024-04-22T07:04:03.801715+0000","last_became_peered":"2024-04-22T07:04:03.801715+0000","last_unstale":"2024-04-22T07:04:04.862265+0000","last_undegraded":"2024-04-22T07:04:04.862265+0000","last_fullsized":"2024-04-22T07:04:04.862265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:29.815068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788145+0000","last_change":"2024-04-22T07:04:02.793270+0000","last_active":"2024-04-22T07:04:04.788145+0000","last_peered":"2024-04-22T07:04:04.788145+0000","last_clean":"2024-04-22T07:04:04.788145+0000","last_became_active":"2024-04-22T07:04:02.793009+0000","last_became_peered":"2024-04-22T07:04:02.793009+0000","last_unstale":"2024-04-22T07:04:04.788145+0000","last_undegraded":"2024-04-22T07:04:04.788145+0000","last_fullsized":"2024-04-22T07:04:04.788145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861459+0000","last_change":"2024-04-22T07:04:03.800175+0000","last_active":"2024-04-22T07:04:04.861459+0000","last_peered":"2024-04-22T07:04:04.861459+0000","last_clean":"2024-04-22T07:04:04.861459+0000","last_became_active":"2024-04-22T07:04:03.799902+0000","last_became_peered":"2024-04-22T07:04:03.799902+0000","last_unstale":"2024-04-22T07:04:04.861459+0000","last_undegraded":"2024-04-22T07:04:04.861459+0000","last_fullsized":"2024-04-22T07:04:04.861459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859806+0000","last_change":"2024-04-22T07:04:02.793643+0000","last_active":"2024-04-22T07:04:04.859806+0000","last_peered":"2024-04-22T07:04:04.859806+0000","last_clean":"2024-04-22T07:04:04.859806+0000","last_became_active":"2024-04-22T07:04:02.793333+0000","last_became_peered":"2024-04-22T07:04:02.793333+0000","last_unstale":"2024-04-22T07:04:04.859806+0000","last_undegraded":"2024-04-22T07:04:04.859806+0000","last_fullsized":"2024-04-22T07:04:04.859806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862019+0000","last_change":"2024-04-22T07:04:03.801821+0000","last_active":"2024-04-22T07:04:04.862019+0000","last_peered":"2024-04-22T07:04:04.862019+0000","last_clean":"2024-04-22T07:04:04.862019+0000","last_became_active":"2024-04-22T07:04:03.801510+0000","last_became_peered":"2024-04-22T07:04:03.801510+0000","last_unstale":"2024-04-22T07:04:04.862019+0000","last_undegraded":"2024-04-22T07:04:04.862019+0000","last_fullsized":"2024-04-22T07:04:04.862019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861241+0000","last_change":"2024-04-22T07:03:59.215285+0000","last_active":"2024-04-22T07:04:04.861241+0000","last_peered":"2024-04-22T07:04:04.861241+0000","last_clean":"2024-04-22T07:04:04.861241+0000","last_became_active":"2024-04-22T07:03:59.214275+0000","last_became_peered":"2024-04-22T07:03:59.214275+0000","last_unstale":"2024-04-22T07:04:04.861241+0000","last_undegraded":"2024-04-22T07:04:04.861241+0000","last_fullsized":"2024-04-22T07:04:04.861241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_clean_scrub_stamp":"2024-04-22T07:03:57.959761+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:55.105073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860226+0000","last_change":"2024-04-22T07:04:02.793384+0000","last_active":"2024-04-22T07:04:04.860226+0000","last_peered":"2024-04-22T07:04:04.860226+0000","last_clean":"2024-04-22T07:04:04.860226+0000","last_became_active":"2024-04-22T07:04:02.793077+0000","last_became_peered":"2024-04-22T07:04:02.793077+0000","last_unstale":"2024-04-22T07:04:04.860226+0000","last_undegraded":"2024-04-22T07:04:04.860226+0000","last_fullsized":"2024-04-22T07:04:04.860226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860779+0000","last_change":"2024-04-22T07:04:03.798134+0000","last_active":"2024-04-22T07:04:04.860779+0000","last_peered":"2024-04-22T07:04:04.860779+0000","last_clean":"2024-04-22T07:04:04.860779+0000","last_became_active":"2024-04-22T07:04:03.797848+0000","last_became_peered":"2024-04-22T07:04:03.797848+0000","last_unstale":"2024-04-22T07:04:04.860779+0000","last_undegraded":"2024-04-22T07:04:04.860779+0000","last_fullsized":"2024-04-22T07:04:04.860779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859966+0000","last_change":"2024-04-22T07:04:02.789591+0000","last_active":"2024-04-22T07:04:04.859966+0000","last_peered":"2024-04-22T07:04:04.859966+0000","last_clean":"2024-04-22T07:04:04.859966+0000","last_became_active":"2024-04-22T07:04:02.788785+0000","last_became_peered":"2024-04-22T07:04:02.788785+0000","last_unstale":"2024-04-22T07:04:04.859966+0000","last_undegraded":"2024-04-22T07:04:04.859966+0000","last_fullsized":"2024-04-22T07:04:04.859966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859611+0000","last_change":"2024-04-22T07:04:03.789757+0000","last_active":"2024-04-22T07:04:04.859611+0000","last_peered":"2024-04-22T07:04:04.859611+0000","last_clean":"2024-04-22T07:04:04.859611+0000","last_became_active":"2024-04-22T07:04:03.789463+0000","last_became_peered":"2024-04-22T07:04:03.789463+0000","last_unstale":"2024-04-22T07:04:04.859611+0000","last_undegraded":"2024-04-22T07:04:04.859611+0000","last_fullsized":"2024-04-22T07:04:04.859611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:16.859332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859989+0000","last_change":"2024-04-22T07:04:02.790600+0000","last_active":"2024-04-22T07:04:04.859989+0000","last_peered":"2024-04-22T07:04:04.859989+0000","last_clean":"2024-04-22T07:04:04.859989+0000","last_became_active":"2024-04-22T07:04:02.790319+0000","last_became_peered":"2024-04-22T07:04:02.790319+0000","last_unstale":"2024-04-22T07:04:04.859989+0000","last_undegraded":"2024-04-22T07:04:04.859989+0000","last_fullsized":"2024-04-22T07:04:04.859989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787840+0000","last_change":"2024-04-22T07:04:03.788231+0000","last_active":"2024-04-22T07:04:04.787840+0000","last_peered":"2024-04-22T07:04:04.787840+0000","last_clean":"2024-04-22T07:04:04.787840+0000","last_became_active":"2024-04-22T07:04:03.787918+0000","last_became_peered":"2024-04-22T07:04:03.787918+0000","last_unstale":"2024-04-22T07:04:04.787840+0000","last_undegraded":"2024-04-22T07:04:04.787840+0000","last_fullsized":"2024-04-22T07:04:04.787840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861257+0000","last_change":"2024-04-22T07:04:02.796245+0000","last_active":"2024-04-22T07:04:04.861257+0000","last_peered":"2024-04-22T07:04:04.861257+0000","last_clean":"2024-04-22T07:04:04.861257+0000","last_became_active":"2024-04-22T07:04:02.795947+0000","last_became_peered":"2024-04-22T07:04:02.795947+0000","last_unstale":"2024-04-22T07:04:04.861257+0000","last_undegraded":"2024-04-22T07:04:04.861257+0000","last_fullsized":"2024-04-22T07:04:04.861257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790274+0000","last_change":"2024-04-22T07:04:03.793330+0000","last_active":"2024-04-22T07:04:04.790274+0000","last_peered":"2024-04-22T07:04:04.790274+0000","last_clean":"2024-04-22T07:04:04.790274+0000","last_became_active":"2024-04-22T07:04:03.793037+0000","last_became_peered":"2024-04-22T07:04:03.793037+0000","last_unstale":"2024-04-22T07:04:04.790274+0000","last_undegraded":"2024-04-22T07:04:04.790274+0000","last_fullsized":"2024-04-22T07:04:04.790274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:13.368853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787950+0000","last_change":"2024-04-22T07:04:02.793498+0000","last_active":"2024-04-22T07:04:04.787950+0000","last_peered":"2024-04-22T07:04:04.787950+0000","last_clean":"2024-04-22T07:04:04.787950+0000","last_became_active":"2024-04-22T07:04:02.793215+0000","last_became_peered":"2024-04-22T07:04:02.793215+0000","last_unstale":"2024-04-22T07:04:04.787950+0000","last_undegraded":"2024-04-22T07:04:04.787950+0000","last_fullsized":"2024-04-22T07:04:04.787950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860866+0000","last_change":"2024-04-22T07:04:03.804152+0000","last_active":"2024-04-22T07:04:04.860866+0000","last_peered":"2024-04-22T07:04:04.860866+0000","last_clean":"2024-04-22T07:04:04.860866+0000","last_became_active":"2024-04-22T07:04:03.801961+0000","last_became_peered":"2024-04-22T07:04:03.801961+0000","last_unstale":"2024-04-22T07:04:04.860866+0000","last_undegraded":"2024-04-22T07:04:04.860866+0000","last_fullsized":"2024-04-22T07:04:04.860866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862105+0000","last_change":"2024-04-22T07:04:02.793445+0000","last_active":"2024-04-22T07:04:04.862105+0000","last_peered":"2024-04-22T07:04:04.862105+0000","last_clean":"2024-04-22T07:04:04.862105+0000","last_became_active":"2024-04-22T07:04:02.793164+0000","last_became_peered":"2024-04-22T07:04:02.793164+0000","last_unstale":"2024-04-22T07:04:04.862105+0000","last_undegraded":"2024-04-22T07:04:04.862105+0000","last_fullsized":"2024-04-22T07:04:04.862105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788013+0000","last_change":"2024-04-22T07:04:03.789589+0000","last_active":"2024-04-22T07:04:04.788013+0000","last_peered":"2024-04-22T07:04:04.788013+0000","last_clean":"2024-04-22T07:04:04.788013+0000","last_became_active":"2024-04-22T07:04:03.789310+0000","last_became_peered":"2024-04-22T07:04:03.789310+0000","last_unstale":"2024-04-22T07:04:04.788013+0000","last_undegraded":"2024-04-22T07:04:04.788013+0000","last_fullsized":"2024-04-22T07:04:04.788013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788077+0000","last_change":"2024-04-22T07:04:02.789243+0000","last_active":"2024-04-22T07:04:04.788077+0000","last_peered":"2024-04-22T07:04:04.788077+0000","last_clean":"2024-04-22T07:04:04.788077+0000","last_became_active":"2024-04-22T07:04:02.788897+0000","last_became_peered":"2024-04-22T07:04:02.788897+0000","last_unstale":"2024-04-22T07:04:04.788077+0000","last_undegraded":"2024-04-22T07:04:04.788077+0000","last_fullsized":"2024-04-22T07:04:04.788077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860272+0000","last_change":"2024-04-22T07:04:03.793656+0000","last_active":"2024-04-22T07:04:04.860272+0000","last_peered":"2024-04-22T07:04:04.860272+0000","last_clean":"2024-04-22T07:04:04.860272+0000","last_became_active":"2024-04-22T07:04:03.793311+0000","last_became_peered":"2024-04-22T07:04:03.793311+0000","last_unstale":"2024-04-22T07:04:04.860272+0000","last_undegraded":"2024-04-22T07:04:04.860272+0000","last_fullsized":"2024-04-22T07:04:04.860272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.783618+0000","last_change":"2024-04-22T07:04:02.786878+0000","last_active":"2024-04-22T07:04:08.783618+0000","last_peered":"2024-04-22T07:04:08.783618+0000","last_clean":"2024-04-22T07:04:08.783618+0000","last_became_active":"2024-04-22T07:04:02.786526+0000","last_became_peered":"2024-04-22T07:04:02.786526+0000","last_unstale":"2024-04-22T07:04:08.783618+0000","last_undegraded":"2024-04-22T07:04:08.783618+0000","last_fullsized":"2024-04-22T07:04:08.783618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860520+0000","last_change":"2024-04-22T07:04:03.791544+0000","last_active":"2024-04-22T07:04:04.860520+0000","last_peered":"2024-04-22T07:04:04.860520+0000","last_clean":"2024-04-22T07:04:04.860520+0000","last_became_active":"2024-04-22T07:04:03.791289+0000","last_became_peered":"2024-04-22T07:04:03.791289+0000","last_unstale":"2024-04-22T07:04:04.860520+0000","last_undegraded":"2024-04-22T07:04:04.860520+0000","last_fullsized":"2024-04-22T07:04:04.860520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859963+0000","last_change":"2024-04-22T07:04:02.795178+0000","last_active":"2024-04-22T07:04:04.859963+0000","last_peered":"2024-04-22T07:04:04.859963+0000","last_clean":"2024-04-22T07:04:04.859963+0000","last_became_active":"2024-04-22T07:04:02.794795+0000","last_became_peered":"2024-04-22T07:04:02.794795+0000","last_unstale":"2024-04-22T07:04:04.859963+0000","last_undegraded":"2024-04-22T07:04:04.859963+0000","last_fullsized":"2024-04-22T07:04:04.859963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790163+0000","last_change":"2024-04-22T07:04:03.788560+0000","last_active":"2024-04-22T07:04:04.790163+0000","last_peered":"2024-04-22T07:04:04.790163+0000","last_clean":"2024-04-22T07:04:04.790163+0000","last_became_active":"2024-04-22T07:04:03.788278+0000","last_became_peered":"2024-04-22T07:04:03.788278+0000","last_unstale":"2024-04-22T07:04:04.790163+0000","last_undegraded":"2024-04-22T07:04:04.790163+0000","last_fullsized":"2024-04-22T07:04:04.790163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:37.492038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787876+0000","last_change":"2024-04-22T07:04:02.788532+0000","last_active":"2024-04-22T07:04:04.787876+0000","last_peered":"2024-04-22T07:04:04.787876+0000","last_clean":"2024-04-22T07:04:04.787876+0000","last_became_active":"2024-04-22T07:04:02.788254+0000","last_became_peered":"2024-04-22T07:04:02.788254+0000","last_unstale":"2024-04-22T07:04:04.787876+0000","last_undegraded":"2024-04-22T07:04:04.787876+0000","last_fullsized":"2024-04-22T07:04:04.787876+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859748+0000","last_change":"2024-04-22T07:04:03.790052+0000","last_active":"2024-04-22T07:04:04.859748+0000","last_peered":"2024-04-22T07:04:04.859748+0000","last_clean":"2024-04-22T07:04:04.859748+0000","last_became_active":"2024-04-22T07:04:03.789351+0000","last_became_peered":"2024-04-22T07:04:03.789351+0000","last_unstale":"2024-04-22T07:04:04.859748+0000","last_undegraded":"2024-04-22T07:04:04.859748+0000","last_fullsized":"2024-04-22T07:04:04.859748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787903+0000","last_change":"2024-04-22T07:04:02.792774+0000","last_active":"2024-04-22T07:04:04.787903+0000","last_peered":"2024-04-22T07:04:04.787903+0000","last_clean":"2024-04-22T07:04:04.787903+0000","last_became_active":"2024-04-22T07:04:02.792282+0000","last_became_peered":"2024-04-22T07:04:02.792282+0000","last_unstale":"2024-04-22T07:04:04.787903+0000","last_undegraded":"2024-04-22T07:04:04.787903+0000","last_fullsized":"2024-04-22T07:04:04.787903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860691+0000","last_change":"2024-04-22T07:04:03.796133+0000","last_active":"2024-04-22T07:04:04.860691+0000","last_peered":"2024-04-22T07:04:04.860691+0000","last_clean":"2024-04-22T07:04:04.860691+0000","last_became_active":"2024-04-22T07:04:03.795854+0000","last_became_peered":"2024-04-22T07:04:03.795854+0000","last_unstale":"2024-04-22T07:04:04.860691+0000","last_undegraded":"2024-04-22T07:04:04.860691+0000","last_fullsized":"2024-04-22T07:04:04.860691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.783241+0000","last_change":"2024-04-22T07:04:02.789099+0000","last_active":"2024-04-22T07:04:08.783241+0000","last_peered":"2024-04-22T07:04:08.783241+0000","last_clean":"2024-04-22T07:04:08.783241+0000","last_became_active":"2024-04-22T07:04:02.788782+0000","last_became_peered":"2024-04-22T07:04:02.788782+0000","last_unstale":"2024-04-22T07:04:08.783241+0000","last_undegraded":"2024-04-22T07:04:08.783241+0000","last_fullsized":"2024-04-22T07:04:08.783241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860112+0000","last_change":"2024-04-22T07:04:03.799845+0000","last_active":"2024-04-22T07:04:04.860112+0000","last_peered":"2024-04-22T07:04:04.860112+0000","last_clean":"2024-04-22T07:04:04.860112+0000","last_became_active":"2024-04-22T07:04:03.799553+0000","last_became_peered":"2024-04-22T07:04:03.799553+0000","last_unstale":"2024-04-22T07:04:04.860112+0000","last_undegraded":"2024-04-22T07:04:04.860112+0000","last_fullsized":"2024-04-22T07:04:04.860112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860699+0000","last_change":"2024-04-22T07:04:03.799918+0000","last_active":"2024-04-22T07:04:04.860699+0000","last_peered":"2024-04-22T07:04:04.860699+0000","last_clean":"2024-04-22T07:04:04.860699+0000","last_became_active":"2024-04-22T07:04:03.797851+0000","last_became_peered":"2024-04-22T07:04:03.797851+0000","last_unstale":"2024-04-22T07:04:04.860699+0000","last_undegraded":"2024-04-22T07:04:04.860699+0000","last_fullsized":"2024-04-22T07:04:04.860699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.782807+0000","last_change":"2024-04-22T07:04:02.793627+0000","last_active":"2024-04-22T07:04:08.782807+0000","last_peered":"2024-04-22T07:04:08.782807+0000","last_clean":"2024-04-22T07:04:08.782807+0000","last_became_active":"2024-04-22T07:04:02.793341+0000","last_became_peered":"2024-04-22T07:04:02.793341+0000","last_unstale":"2024-04-22T07:04:08.782807+0000","last_undegraded":"2024-04-22T07:04:08.782807+0000","last_fullsized":"2024-04-22T07:04:08.782807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:21:07.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861294+0000","last_change":"2024-04-22T07:04:03.797768+0000","last_active":"2024-04-22T07:04:04.861294+0000","last_peered":"2024-04-22T07:04:04.861294+0000","last_clean":"2024-04-22T07:04:04.861294+0000","last_became_active":"2024-04-22T07:04:03.797437+0000","last_became_peered":"2024-04-22T07:04:03.797437+0000","last_unstale":"2024-04-22T07:04:04.861294+0000","last_undegraded":"2024-04-22T07:04:04.861294+0000","last_fullsized":"2024-04-22T07:04:04.861294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.783111+0000","last_change":"2024-04-22T07:04:02.789875+0000","last_active":"2024-04-22T07:04:08.783111+0000","last_peered":"2024-04-22T07:04:08.783111+0000","last_clean":"2024-04-22T07:04:08.783111+0000","last_became_active":"2024-04-22T07:04:02.788989+0000","last_became_peered":"2024-04-22T07:04:02.788989+0000","last_unstale":"2024-04-22T07:04:08.783111+0000","last_undegraded":"2024-04-22T07:04:08.783111+0000","last_fullsized":"2024-04-22T07:04:08.783111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860096+0000","last_change":"2024-04-22T07:04:03.797671+0000","last_active":"2024-04-22T07:04:04.860096+0000","last_peered":"2024-04-22T07:04:04.860096+0000","last_clean":"2024-04-22T07:04:04.860096+0000","last_became_active":"2024-04-22T07:04:03.797399+0000","last_became_peered":"2024-04-22T07:04:03.797399+0000","last_unstale":"2024-04-22T07:04:04.860096+0000","last_undegraded":"2024-04-22T07:04:04.860096+0000","last_fullsized":"2024-04-22T07:04:04.860096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860445+0000","last_change":"2024-04-22T07:04:02.796141+0000","last_active":"2024-04-22T07:04:04.860445+0000","last_peered":"2024-04-22T07:04:04.860445+0000","last_clean":"2024-04-22T07:04:04.860445+0000","last_became_active":"2024-04-22T07:04:02.795821+0000","last_became_peered":"2024-04-22T07:04:02.795821+0000","last_unstale":"2024-04-22T07:04:04.860445+0000","last_undegraded":"2024-04-22T07:04:04.860445+0000","last_fullsized":"2024-04-22T07:04:04.860445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860371+0000","last_change":"2024-04-22T07:04:03.796729+0000","last_active":"2024-04-22T07:04:04.860371+0000","last_peered":"2024-04-22T07:04:04.860371+0000","last_clean":"2024-04-22T07:04:04.860371+0000","last_became_active":"2024-04-22T07:04:03.796486+0000","last_became_peered":"2024-04-22T07:04:03.796486+0000","last_unstale":"2024-04-22T07:04:04.860371+0000","last_undegraded":"2024-04-22T07:04:04.860371+0000","last_fullsized":"2024-04-22T07:04:04.860371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.783665+0000","last_change":"2024-04-22T07:04:02.793214+0000","last_active":"2024-04-22T07:04:08.783665+0000","last_peered":"2024-04-22T07:04:08.783665+0000","last_clean":"2024-04-22T07:04:08.783665+0000","last_became_active":"2024-04-22T07:04:02.792922+0000","last_became_peered":"2024-04-22T07:04:02.792922+0000","last_unstale":"2024-04-22T07:04:08.783665+0000","last_undegraded":"2024-04-22T07:04:08.783665+0000","last_fullsized":"2024-04-22T07:04:08.783665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:14.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:14.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.958+0000 7f111bfd5640 1 Processor -- start 2024-04-22T07:04:14.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.959+0000 7f111bfd5640 1 -- start start 2024-04-22T07:04:14.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.959+0000 7f111bfd5640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f11140700c0 0x7f11140704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.959+0000 7f111bfd5640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11140c9600 0x7f111415c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.959+0000 7f111bfd5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11140c55e0 con 0x7f111415cc80 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.959+0000 7f111bfd5640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f111415bfd0 con 0x7f11140700c0 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.959+0000 7f111bfd5640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f11140c3520 con 0x7f11140c9600 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f111a572640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f111415cc80 0x7f1114158a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46780/0 (socket says 172.21.15.92:46780) 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f111a572640 1 -- 172.21.15.92:0/2979108786 learned_addr learned my addr 172.21.15.92:0/2979108786 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1119570640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11140c9600 0x7f111415c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 168342062 0 0) 0x7f11140c55e0 con 0x7f111415cc80 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1119d71640 1 --2- 172.21.15.92:0/2979108786 >> v2:172.21.15.145:3300/0 conn(0x7f11140700c0 0x7f11140704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1100003520 con 0x7f111415cc80 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2638106973 0 0) 0x7f1100003520 con 0x7f111415cc80 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11140c9600 msgr2=0x7f111415c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11140c9600 0x7f111415c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 >> v2:172.21.15.145:3300/0 conn(0x7f11140700c0 msgr2=0x7f11140704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 --2- 172.21.15.92:0/2979108786 >> v2:172.21.15.145:3300/0 conn(0x7f11140700c0 0x7f11140704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11140c3800 con 0x7f111415cc80 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f110c0019f0 con 0x7f111415cc80 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1119570640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11140c9600 0x7f111415c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 >> v1:172.21.15.92:6789/0 conn(0x7f111415cc80 legacy=0x7f1114158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11000039b0 0x7f1100003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1100004800 0x7f1100004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11140c9600 0x7f11000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f111415bfd0 con 0x7f11000039b0 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f11140c3520 con 0x7f1100004800 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f110000b9c0 con 0x7f11140c9600 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1119d71640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11000039b0 0x7f1100003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f111a572640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11140c9600 0x7f11000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1119570640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1100004800 0x7f1100004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.960+0000 7f1119d71640 1 -- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11140c9600 msgr2=0x7f11000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.961+0000 7f1119d71640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11140c9600 0x7f11000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.961+0000 7f1119d71640 1 -- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1100004800 msgr2=0x7f1100004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.961+0000 7f1119d71640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1100004800 0x7f1100004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.961+0000 7f1119d71640 1 -- 172.21.15.92:0/2979108786 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1100003520 con 0x7f11000039b0 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.961+0000 7f111a572640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11140c9600 0x7f11000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.961+0000 7f1119570640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1100004800 0x7f1100004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.961+0000 7f1119d71640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11000039b0 0x7f1100003dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1110009ac0 tx=0x7f1100005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595518dade52a289 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.961+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110039070 con 0x7f11000039b0 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.961+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f111002c710 con 0x7f11000039b0 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.961+0000 7f1118d6f640 1 -- 172.21.15.92:0/2979108786 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110009690 con 0x7f11000039b0 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.962+0000 7f111bfd5640 1 -- 172.21.15.92:0/2979108786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11000039b0 msgr2=0x7f1100003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.962+0000 7f111bfd5640 1 --2- 172.21.15.92:0/2979108786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11000039b0 0x7f1100003dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1110009ac0 tx=0x7f1100005830 comp rx=0 tx=0).stop 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.962+0000 7f111a572640 1 -- 172.21.15.92:0/2979108786 reap_dead start 2024-04-22T07:04:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.962+0000 7f111bfd5640 1 -- 172.21.15.92:0/2979108786 shutdown_connections 2024-04-22T07:04:14.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.962+0000 7f111bfd5640 1 -- 172.21.15.92:0/2979108786 >> 172.21.15.92:0/2979108786 conn(0x7f111406db80 msgr2=0x7f111406df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.962+0000 7f111bfd5640 1 -- 172.21.15.92:0/2979108786 shutdown_connections 2024-04-22T07:04:14.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.962+0000 7f111bfd5640 1 -- 172.21.15.92:0/2979108786 wait complete. 2024-04-22T07:04:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.963+0000 7f111bfd5640 1 Processor -- start 2024-04-22T07:04:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.963+0000 7f111bfd5640 1 -- start start 2024-04-22T07:04:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111bfd5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11000039b0 0x7f111414f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111bfd5640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11141f1710 0x7f11140b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111bfd5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f111414d130 0x7f111414d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111bfd5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f11140ca570 con 0x7f111414d130 2024-04-22T07:04:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111bfd5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f111415c220 con 0x7f11000039b0 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111bfd5640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f11140c5230 con 0x7f11141f1710 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111a572640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f111414d130 0x7f111414d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111a572640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f111414d130 0x7f111414d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36516/0 (socket says 172.21.15.92:36516) 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111a572640 1 -- 172.21.15.92:0/1440827963 learned_addr learned my addr 172.21.15.92:0/1440827963 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111a572640 1 -- 172.21.15.92:0/1440827963 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11141f1710 msgr2=0x7f11140b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111a572640 1 --2- 172.21.15.92:0/1440827963 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11141f1710 0x7f11140b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111a572640 1 -- 172.21.15.92:0/1440827963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11000039b0 msgr2=0x7f111414f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111a572640 1 --2- 172.21.15.92:0/1440827963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11000039b0 0x7f111414f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111a572640 1 -- 172.21.15.92:0/1440827963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11141e3eb0 con 0x7f111414d130 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111a572640 1 --2- 172.21.15.92:0/1440827963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f111414d130 0x7f111414d5c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f11140c4620 tx=0x7f110c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f110a7fc640 1 -- 172.21.15.92:0/1440827963 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c00ed40 con 0x7f111414d130 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f110a7fc640 1 -- 172.21.15.92:0/1440827963 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c018070 con 0x7f111414d130 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111bfd5640 1 -- 172.21.15.92:0/1440827963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111414db00 con 0x7f111414d130 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f110a7fc640 1 -- 172.21.15.92:0/1440827963 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c013640 con 0x7f111414d130 2024-04-22T07:04:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.964+0000 7f111bfd5640 1 -- 172.21.15.92:0/1440827963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11141e32a0 con 0x7f111414d130 2024-04-22T07:04:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.965+0000 7f111bfd5640 1 -- 172.21.15.92:0/1440827963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1114068b00 con 0x7f111414d130 2024-04-22T07:04:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.966+0000 7f110a7fc640 1 -- 172.21.15.92:0/1440827963 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f110c0137e0 con 0x7f111414d130 2024-04-22T07:04:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.966+0000 7f110a7fc640 1 --2- 172.21.15.92:0/1440827963 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f10f806c8f0 0x7f10f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.967+0000 7f1119d71640 1 --2- 172.21.15.92:0/1440827963 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f10f806c8f0 0x7f10f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.967+0000 7f110a7fc640 1 -- 172.21.15.92:0/1440827963 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f110c08bea0 con 0x7f111414d130 2024-04-22T07:04:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.967+0000 7f1119d71640 1 --2- 172.21.15.92:0/1440827963 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f10f806c8f0 0x7f10f806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1110036000 tx=0x7f111002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:14.971+0000 7f110a7fc640 1 -- 172.21.15.92:0/1440827963 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f110c05a360 con 0x7f111414d130 2024-04-22T07:04:15.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.148+0000 7f111bfd5640 1 -- 172.21.15.92:0/1440827963 --> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f111414ce90 con 0x7f10f806c8f0 2024-04-22T07:04:15.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.154+0000 7f110a7fc640 1 -- 172.21.15.92:0/1440827963 <== mgr.4107 v2:172.21.15.92:6832/479317737 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f111414ce90 con 0x7f10f806c8f0 2024-04-22T07:04:15.154 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:15.155 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-22T07:04:15.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.157+0000 7f111bfd5640 1 -- 172.21.15.92:0/1440827963 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f10f806c8f0 msgr2=0x7f10f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.157+0000 7f111bfd5640 1 --2- 172.21.15.92:0/1440827963 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f10f806c8f0 0x7f10f806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1110036000 tx=0x7f111002e000 comp rx=0 tx=0).stop 2024-04-22T07:04:15.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.157+0000 7f111bfd5640 1 -- 172.21.15.92:0/1440827963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f111414d130 msgr2=0x7f111414d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.157+0000 7f111bfd5640 1 --2- 172.21.15.92:0/1440827963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f111414d130 0x7f111414d5c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f11140c4620 tx=0x7f110c00e530 comp rx=0 tx=0).stop 2024-04-22T07:04:15.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.157+0000 7f111bfd5640 1 -- 172.21.15.92:0/1440827963 shutdown_connections 2024-04-22T07:04:15.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.157+0000 7f111bfd5640 1 --2- 172.21.15.92:0/1440827963 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f10f806c8f0 0x7f10f806edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.157+0000 7f111bfd5640 1 --2- 172.21.15.92:0/1440827963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f111414d130 0x7f111414d5c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.157+0000 7f111bfd5640 1 --2- 172.21.15.92:0/1440827963 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11141f1710 0x7f11140b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.157+0000 7f111bfd5640 1 --2- 172.21.15.92:0/1440827963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11000039b0 0x7f111414f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.157+0000 7f111bfd5640 1 -- 172.21.15.92:0/1440827963 >> 172.21.15.92:0/1440827963 conn(0x7f111406db80 msgr2=0x7f11140cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.157+0000 7f111bfd5640 1 -- 172.21.15.92:0/1440827963 shutdown_connections 2024-04-22T07:04:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.157+0000 7f111bfd5640 1 -- 172.21.15.92:0/1440827963 wait complete. 2024-04-22T07:04:15.165 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-22T07:04:13.934904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218952,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755768,"statfs":{"total":773094113280,"available":772869906432,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001905"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862171+0000","last_change":"2024-04-22T07:04:03.796924+0000","last_active":"2024-04-22T07:04:04.862171+0000","last_peered":"2024-04-22T07:04:04.862171+0000","last_clean":"2024-04-22T07:04:04.862171+0000","last_became_active":"2024-04-22T07:04:03.796652+0000","last_became_peered":"2024-04-22T07:04:03.796652+0000","last_unstale":"2024-04-22T07:04:04.862171+0000","last_undegraded":"2024-04-22T07:04:04.862171+0000","last_fullsized":"2024-04-22T07:04:04.862171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:01.867906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788117+0000","last_change":"2024-04-22T07:04:03.793652+0000","last_active":"2024-04-22T07:04:04.788117+0000","last_peered":"2024-04-22T07:04:04.788117+0000","last_clean":"2024-04-22T07:04:04.788117+0000","last_became_active":"2024-04-22T07:04:03.793397+0000","last_became_peered":"2024-04-22T07:04:03.793397+0000","last_unstale":"2024-04-22T07:04:04.788117+0000","last_undegraded":"2024-04-22T07:04:04.788117+0000","last_fullsized":"2024-04-22T07:04:04.788117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859908+0000","last_change":"2024-04-22T07:04:03.788534+0000","last_active":"2024-04-22T07:04:04.859908+0000","last_peered":"2024-04-22T07:04:04.859908+0000","last_clean":"2024-04-22T07:04:04.859908+0000","last_became_active":"2024-04-22T07:04:03.787747+0000","last_became_peered":"2024-04-22T07:04:03.787747+0000","last_unstale":"2024-04-22T07:04:04.859908+0000","last_undegraded":"2024-04-22T07:04:04.859908+0000","last_fullsized":"2024-04-22T07:04:04.859908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861337+0000","last_change":"2024-04-22T07:04:03.800150+0000","last_active":"2024-04-22T07:04:04.861337+0000","last_peered":"2024-04-22T07:04:04.861337+0000","last_clean":"2024-04-22T07:04:04.861337+0000","last_became_active":"2024-04-22T07:04:03.799887+0000","last_became_peered":"2024-04-22T07:04:03.799887+0000","last_unstale":"2024-04-22T07:04:04.861337+0000","last_undegraded":"2024-04-22T07:04:04.861337+0000","last_fullsized":"2024-04-22T07:04:04.861337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861727+0000","last_change":"2024-04-22T07:04:03.798482+0000","last_active":"2024-04-22T07:04:04.861727+0000","last_peered":"2024-04-22T07:04:04.861727+0000","last_clean":"2024-04-22T07:04:04.861727+0000","last_became_active":"2024-04-22T07:04:03.798205+0000","last_became_peered":"2024-04-22T07:04:03.798205+0000","last_unstale":"2024-04-22T07:04:04.861727+0000","last_undegraded":"2024-04-22T07:04:04.861727+0000","last_fullsized":"2024-04-22T07:04:04.861727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861044+0000","last_change":"2024-04-22T07:04:03.797373+0000","last_active":"2024-04-22T07:04:04.861044+0000","last_peered":"2024-04-22T07:04:04.861044+0000","last_clean":"2024-04-22T07:04:04.861044+0000","last_became_active":"2024-04-22T07:04:03.797090+0000","last_became_peered":"2024-04-22T07:04:03.797090+0000","last_unstale":"2024-04-22T07:04:04.861044+0000","last_undegraded":"2024-04-22T07:04:04.861044+0000","last_fullsized":"2024-04-22T07:04:04.861044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861437+0000","last_change":"2024-04-22T07:04:03.793725+0000","last_active":"2024-04-22T07:04:04.861437+0000","last_peered":"2024-04-22T07:04:04.861437+0000","last_clean":"2024-04-22T07:04:04.861437+0000","last_became_active":"2024-04-22T07:04:03.793418+0000","last_became_peered":"2024-04-22T07:04:03.793418+0000","last_unstale":"2024-04-22T07:04:04.861437+0000","last_undegraded":"2024-04-22T07:04:04.861437+0000","last_fullsized":"2024-04-22T07:04:04.861437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859500+0000","last_change":"2024-04-22T07:04:03.795377+0000","last_active":"2024-04-22T07:04:04.859500+0000","last_peered":"2024-04-22T07:04:04.859500+0000","last_clean":"2024-04-22T07:04:04.859500+0000","last_became_active":"2024-04-22T07:04:03.795106+0000","last_became_peered":"2024-04-22T07:04:03.795106+0000","last_unstale":"2024-04-22T07:04:04.859500+0000","last_undegraded":"2024-04-22T07:04:04.859500+0000","last_fullsized":"2024-04-22T07:04:04.859500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862209+0000","last_change":"2024-04-22T07:04:03.797692+0000","last_active":"2024-04-22T07:04:04.862209+0000","last_peered":"2024-04-22T07:04:04.862209+0000","last_clean":"2024-04-22T07:04:04.862209+0000","last_became_active":"2024-04-22T07:04:03.797396+0000","last_became_peered":"2024-04-22T07:04:03.797396+0000","last_unstale":"2024-04-22T07:04:04.862209+0000","last_undegraded":"2024-04-22T07:04:04.862209+0000","last_fullsized":"2024-04-22T07:04:04.862209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:39.068857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860742+0000","last_change":"2024-04-22T07:04:03.797228+0000","last_active":"2024-04-22T07:04:04.860742+0000","last_peered":"2024-04-22T07:04:04.860742+0000","last_clean":"2024-04-22T07:04:04.860742+0000","last_became_active":"2024-04-22T07:04:03.796614+0000","last_became_peered":"2024-04-22T07:04:03.796614+0000","last_unstale":"2024-04-22T07:04:04.860742+0000","last_undegraded":"2024-04-22T07:04:04.860742+0000","last_fullsized":"2024-04-22T07:04:04.860742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860497+0000","last_change":"2024-04-22T07:04:03.799434+0000","last_active":"2024-04-22T07:04:04.860497+0000","last_peered":"2024-04-22T07:04:04.860497+0000","last_clean":"2024-04-22T07:04:04.860497+0000","last_became_active":"2024-04-22T07:04:03.799173+0000","last_became_peered":"2024-04-22T07:04:03.799173+0000","last_unstale":"2024-04-22T07:04:04.860497+0000","last_undegraded":"2024-04-22T07:04:04.860497+0000","last_fullsized":"2024-04-22T07:04:04.860497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790488+0000","last_change":"2024-04-22T07:04:03.800035+0000","last_active":"2024-04-22T07:04:04.790488+0000","last_peered":"2024-04-22T07:04:04.790488+0000","last_clean":"2024-04-22T07:04:04.790488+0000","last_became_active":"2024-04-22T07:04:03.799646+0000","last_became_peered":"2024-04-22T07:04:03.799646+0000","last_unstale":"2024-04-22T07:04:04.790488+0000","last_undegraded":"2024-04-22T07:04:04.790488+0000","last_fullsized":"2024-04-22T07:04:04.790488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:17.287068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860651+0000","last_change":"2024-04-22T07:04:03.797324+0000","last_active":"2024-04-22T07:04:04.860651+0000","last_peered":"2024-04-22T07:04:04.860651+0000","last_clean":"2024-04-22T07:04:04.860651+0000","last_became_active":"2024-04-22T07:04:03.797039+0000","last_became_peered":"2024-04-22T07:04:03.797039+0000","last_unstale":"2024-04-22T07:04:04.860651+0000","last_undegraded":"2024-04-22T07:04:04.860651+0000","last_fullsized":"2024-04-22T07:04:04.860651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859720+0000","last_change":"2024-04-22T07:04:03.790732+0000","last_active":"2024-04-22T07:04:04.859720+0000","last_peered":"2024-04-22T07:04:04.859720+0000","last_clean":"2024-04-22T07:04:04.859720+0000","last_became_active":"2024-04-22T07:04:03.788340+0000","last_became_peered":"2024-04-22T07:04:03.788340+0000","last_unstale":"2024-04-22T07:04:04.859720+0000","last_undegraded":"2024-04-22T07:04:04.859720+0000","last_fullsized":"2024-04-22T07:04:04.859720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787860+0000","last_change":"2024-04-22T07:04:03.799368+0000","last_active":"2024-04-22T07:04:04.787860+0000","last_peered":"2024-04-22T07:04:04.787860+0000","last_clean":"2024-04-22T07:04:04.787860+0000","last_became_active":"2024-04-22T07:04:03.799067+0000","last_became_peered":"2024-04-22T07:04:03.799067+0000","last_unstale":"2024-04-22T07:04:04.787860+0000","last_undegraded":"2024-04-22T07:04:04.787860+0000","last_fullsized":"2024-04-22T07:04:04.787860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860988+0000","last_change":"2024-04-22T07:04:03.802947+0000","last_active":"2024-04-22T07:04:04.860988+0000","last_peered":"2024-04-22T07:04:04.860988+0000","last_clean":"2024-04-22T07:04:04.860988+0000","last_became_active":"2024-04-22T07:04:03.802493+0000","last_became_peered":"2024-04-22T07:04:03.802493+0000","last_unstale":"2024-04-22T07:04:04.860988+0000","last_undegraded":"2024-04-22T07:04:04.860988+0000","last_fullsized":"2024-04-22T07:04:04.860988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860714+0000","last_change":"2024-04-22T07:04:03.795581+0000","last_active":"2024-04-22T07:04:04.860714+0000","last_peered":"2024-04-22T07:04:04.860714+0000","last_clean":"2024-04-22T07:04:04.860714+0000","last_became_active":"2024-04-22T07:04:03.795292+0000","last_became_peered":"2024-04-22T07:04:03.795292+0000","last_unstale":"2024-04-22T07:04:04.860714+0000","last_undegraded":"2024-04-22T07:04:04.860714+0000","last_fullsized":"2024-04-22T07:04:04.860714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859993+0000","last_change":"2024-04-22T07:04:03.796132+0000","last_active":"2024-04-22T07:04:04.859993+0000","last_peered":"2024-04-22T07:04:04.859993+0000","last_clean":"2024-04-22T07:04:04.859993+0000","last_became_active":"2024-04-22T07:04:03.795069+0000","last_became_peered":"2024-04-22T07:04:03.795069+0000","last_unstale":"2024-04-22T07:04:04.859993+0000","last_undegraded":"2024-04-22T07:04:04.859993+0000","last_fullsized":"2024-04-22T07:04:04.859993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:08.934970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862125+0000","last_change":"2024-04-22T07:04:03.794552+0000","last_active":"2024-04-22T07:04:04.862125+0000","last_peered":"2024-04-22T07:04:04.862125+0000","last_clean":"2024-04-22T07:04:04.862125+0000","last_became_active":"2024-04-22T07:04:03.794046+0000","last_became_peered":"2024-04-22T07:04:03.794046+0000","last_unstale":"2024-04-22T07:04:04.862125+0000","last_undegraded":"2024-04-22T07:04:04.862125+0000","last_fullsized":"2024-04-22T07:04:04.862125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860703+0000","last_change":"2024-04-22T07:04:03.798686+0000","last_active":"2024-04-22T07:04:04.860703+0000","last_peered":"2024-04-22T07:04:04.860703+0000","last_clean":"2024-04-22T07:04:04.860703+0000","last_became_active":"2024-04-22T07:04:03.798104+0000","last_became_peered":"2024-04-22T07:04:03.798104+0000","last_unstale":"2024-04-22T07:04:04.860703+0000","last_undegraded":"2024-04-22T07:04:04.860703+0000","last_fullsized":"2024-04-22T07:04:04.860703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860494+0000","last_change":"2024-04-22T07:04:03.794215+0000","last_active":"2024-04-22T07:04:04.860494+0000","last_peered":"2024-04-22T07:04:04.860494+0000","last_clean":"2024-04-22T07:04:04.860494+0000","last_became_active":"2024-04-22T07:04:03.793990+0000","last_became_peered":"2024-04-22T07:04:03.793990+0000","last_unstale":"2024-04-22T07:04:04.860494+0000","last_undegraded":"2024-04-22T07:04:04.860494+0000","last_fullsized":"2024-04-22T07:04:04.860494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861162+0000","last_change":"2024-04-22T07:04:03.798539+0000","last_active":"2024-04-22T07:04:04.861162+0000","last_peered":"2024-04-22T07:04:04.861162+0000","last_clean":"2024-04-22T07:04:04.861162+0000","last_became_active":"2024-04-22T07:04:03.797713+0000","last_became_peered":"2024-04-22T07:04:03.797713+0000","last_unstale":"2024-04-22T07:04:04.861162+0000","last_undegraded":"2024-04-22T07:04:04.861162+0000","last_fullsized":"2024-04-22T07:04:04.861162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859945+0000","last_change":"2024-04-22T07:04:03.798610+0000","last_active":"2024-04-22T07:04:04.859945+0000","last_peered":"2024-04-22T07:04:04.859945+0000","last_clean":"2024-04-22T07:04:04.859945+0000","last_became_active":"2024-04-22T07:04:03.797981+0000","last_became_peered":"2024-04-22T07:04:03.797981+0000","last_unstale":"2024-04-22T07:04:04.859945+0000","last_undegraded":"2024-04-22T07:04:04.859945+0000","last_fullsized":"2024-04-22T07:04:04.859945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861507+0000","last_change":"2024-04-22T07:04:03.802742+0000","last_active":"2024-04-22T07:04:04.861507+0000","last_peered":"2024-04-22T07:04:04.861507+0000","last_clean":"2024-04-22T07:04:04.861507+0000","last_became_active":"2024-04-22T07:04:03.802127+0000","last_became_peered":"2024-04-22T07:04:03.802127+0000","last_unstale":"2024-04-22T07:04:04.861507+0000","last_undegraded":"2024-04-22T07:04:04.861507+0000","last_fullsized":"2024-04-22T07:04:04.861507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860138+0000","last_change":"2024-04-22T07:04:03.798789+0000","last_active":"2024-04-22T07:04:04.860138+0000","last_peered":"2024-04-22T07:04:04.860138+0000","last_clean":"2024-04-22T07:04:04.860138+0000","last_became_active":"2024-04-22T07:04:03.798505+0000","last_became_peered":"2024-04-22T07:04:03.798505+0000","last_unstale":"2024-04-22T07:04:04.860138+0000","last_undegraded":"2024-04-22T07:04:04.860138+0000","last_fullsized":"2024-04-22T07:04:04.860138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860919+0000","last_change":"2024-04-22T07:04:03.794111+0000","last_active":"2024-04-22T07:04:04.860919+0000","last_peered":"2024-04-22T07:04:04.860919+0000","last_clean":"2024-04-22T07:04:04.860919+0000","last_became_active":"2024-04-22T07:04:03.793682+0000","last_became_peered":"2024-04-22T07:04:03.793682+0000","last_unstale":"2024-04-22T07:04:04.860919+0000","last_undegraded":"2024-04-22T07:04:04.860919+0000","last_fullsized":"2024-04-22T07:04:04.860919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788083+0000","last_change":"2024-04-22T07:04:03.790167+0000","last_active":"2024-04-22T07:04:04.788083+0000","last_peered":"2024-04-22T07:04:04.788083+0000","last_clean":"2024-04-22T07:04:04.788083+0000","last_became_active":"2024-04-22T07:04:03.789873+0000","last_became_peered":"2024-04-22T07:04:03.789873+0000","last_unstale":"2024-04-22T07:04:04.788083+0000","last_undegraded":"2024-04-22T07:04:04.788083+0000","last_fullsized":"2024-04-22T07:04:04.788083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859954+0000","last_change":"2024-04-22T07:04:03.798713+0000","last_active":"2024-04-22T07:04:04.859954+0000","last_peered":"2024-04-22T07:04:04.859954+0000","last_clean":"2024-04-22T07:04:04.859954+0000","last_became_active":"2024-04-22T07:04:03.798344+0000","last_became_peered":"2024-04-22T07:04:03.798344+0000","last_unstale":"2024-04-22T07:04:04.859954+0000","last_undegraded":"2024-04-22T07:04:04.859954+0000","last_fullsized":"2024-04-22T07:04:04.859954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860634+0000","last_change":"2024-04-22T07:04:03.795452+0000","last_active":"2024-04-22T07:04:04.860634+0000","last_peered":"2024-04-22T07:04:04.860634+0000","last_clean":"2024-04-22T07:04:04.860634+0000","last_became_active":"2024-04-22T07:04:03.794970+0000","last_became_peered":"2024-04-22T07:04:03.794970+0000","last_unstale":"2024-04-22T07:04:04.860634+0000","last_undegraded":"2024-04-22T07:04:04.860634+0000","last_fullsized":"2024-04-22T07:04:04.860634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861203+0000","last_change":"2024-04-22T07:04:03.799581+0000","last_active":"2024-04-22T07:04:04.861203+0000","last_peered":"2024-04-22T07:04:04.861203+0000","last_clean":"2024-04-22T07:04:04.861203+0000","last_became_active":"2024-04-22T07:04:03.799328+0000","last_became_peered":"2024-04-22T07:04:03.799328+0000","last_unstale":"2024-04-22T07:04:04.861203+0000","last_undegraded":"2024-04-22T07:04:04.861203+0000","last_fullsized":"2024-04-22T07:04:04.861203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859944+0000","last_change":"2024-04-22T07:04:03.796830+0000","last_active":"2024-04-22T07:04:04.859944+0000","last_peered":"2024-04-22T07:04:04.859944+0000","last_clean":"2024-04-22T07:04:04.859944+0000","last_became_active":"2024-04-22T07:04:03.796551+0000","last_became_peered":"2024-04-22T07:04:03.796551+0000","last_unstale":"2024-04-22T07:04:04.859944+0000","last_undegraded":"2024-04-22T07:04:04.859944+0000","last_fullsized":"2024-04-22T07:04:04.859944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860101+0000","last_change":"2024-04-22T07:04:03.795320+0000","last_active":"2024-04-22T07:04:04.860101+0000","last_peered":"2024-04-22T07:04:04.860101+0000","last_clean":"2024-04-22T07:04:04.860101+0000","last_became_active":"2024-04-22T07:04:03.795025+0000","last_became_peered":"2024-04-22T07:04:03.795025+0000","last_unstale":"2024-04-22T07:04:04.860101+0000","last_undegraded":"2024-04-22T07:04:04.860101+0000","last_fullsized":"2024-04-22T07:04:04.860101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790119+0000","last_change":"2024-04-22T07:04:03.788859+0000","last_active":"2024-04-22T07:04:04.790119+0000","last_peered":"2024-04-22T07:04:04.790119+0000","last_clean":"2024-04-22T07:04:04.790119+0000","last_became_active":"2024-04-22T07:04:03.788573+0000","last_became_peered":"2024-04-22T07:04:03.788573+0000","last_unstale":"2024-04-22T07:04:04.790119+0000","last_undegraded":"2024-04-22T07:04:04.790119+0000","last_fullsized":"2024-04-22T07:04:04.790119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:36.716689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860730+0000","last_change":"2024-04-22T07:04:02.795724+0000","last_active":"2024-04-22T07:04:04.860730+0000","last_peered":"2024-04-22T07:04:04.860730+0000","last_clean":"2024-04-22T07:04:04.860730+0000","last_became_active":"2024-04-22T07:04:02.795371+0000","last_became_peered":"2024-04-22T07:04:02.795371+0000","last_unstale":"2024-04-22T07:04:04.860730+0000","last_undegraded":"2024-04-22T07:04:04.860730+0000","last_fullsized":"2024-04-22T07:04:04.860730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860689+0000","last_change":"2024-04-22T07:04:03.787030+0000","last_active":"2024-04-22T07:04:04.860689+0000","last_peered":"2024-04-22T07:04:04.860689+0000","last_clean":"2024-04-22T07:04:04.860689+0000","last_became_active":"2024-04-22T07:04:03.786357+0000","last_became_peered":"2024-04-22T07:04:03.786357+0000","last_unstale":"2024-04-22T07:04:04.860689+0000","last_undegraded":"2024-04-22T07:04:04.860689+0000","last_fullsized":"2024-04-22T07:04:04.860689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860368+0000","last_change":"2024-04-22T07:04:02.791429+0000","last_active":"2024-04-22T07:04:04.860368+0000","last_peered":"2024-04-22T07:04:04.860368+0000","last_clean":"2024-04-22T07:04:04.860368+0000","last_became_active":"2024-04-22T07:04:02.791114+0000","last_became_peered":"2024-04-22T07:04:02.791114+0000","last_unstale":"2024-04-22T07:04:04.860368+0000","last_undegraded":"2024-04-22T07:04:04.860368+0000","last_fullsized":"2024-04-22T07:04:04.860368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860040+0000","last_change":"2024-04-22T07:04:02.789531+0000","last_active":"2024-04-22T07:04:04.860040+0000","last_peered":"2024-04-22T07:04:04.860040+0000","last_clean":"2024-04-22T07:04:04.860040+0000","last_became_active":"2024-04-22T07:04:02.789248+0000","last_became_peered":"2024-04-22T07:04:02.789248+0000","last_unstale":"2024-04-22T07:04:04.860040+0000","last_undegraded":"2024-04-22T07:04:04.860040+0000","last_fullsized":"2024-04-22T07:04:04.860040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:06:43.031782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859624+0000","last_change":"2024-04-22T07:04:03.790142+0000","last_active":"2024-04-22T07:04:04.859624+0000","last_peered":"2024-04-22T07:04:04.859624+0000","last_clean":"2024-04-22T07:04:04.859624+0000","last_became_active":"2024-04-22T07:04:03.789869+0000","last_became_peered":"2024-04-22T07:04:03.789869+0000","last_unstale":"2024-04-22T07:04:04.859624+0000","last_undegraded":"2024-04-22T07:04:04.859624+0000","last_fullsized":"2024-04-22T07:04:04.859624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861050+0000","last_change":"2024-04-22T07:04:02.790537+0000","last_active":"2024-04-22T07:04:04.861050+0000","last_peered":"2024-04-22T07:04:04.861050+0000","last_clean":"2024-04-22T07:04:04.861050+0000","last_became_active":"2024-04-22T07:04:02.789072+0000","last_became_peered":"2024-04-22T07:04:02.789072+0000","last_unstale":"2024-04-22T07:04:04.861050+0000","last_undegraded":"2024-04-22T07:04:04.861050+0000","last_fullsized":"2024-04-22T07:04:04.861050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787835+0000","last_change":"2024-04-22T07:04:03.791239+0000","last_active":"2024-04-22T07:04:04.787835+0000","last_peered":"2024-04-22T07:04:04.787835+0000","last_clean":"2024-04-22T07:04:04.787835+0000","last_became_active":"2024-04-22T07:04:03.790515+0000","last_became_peered":"2024-04-22T07:04:03.790515+0000","last_unstale":"2024-04-22T07:04:04.787835+0000","last_undegraded":"2024-04-22T07:04:04.787835+0000","last_fullsized":"2024-04-22T07:04:04.787835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860666+0000","last_change":"2024-04-22T07:04:02.793582+0000","last_active":"2024-04-22T07:04:04.860666+0000","last_peered":"2024-04-22T07:04:04.860666+0000","last_clean":"2024-04-22T07:04:04.860666+0000","last_became_active":"2024-04-22T07:04:02.793281+0000","last_became_peered":"2024-04-22T07:04:02.793281+0000","last_unstale":"2024-04-22T07:04:04.860666+0000","last_undegraded":"2024-04-22T07:04:04.860666+0000","last_fullsized":"2024-04-22T07:04:04.860666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860510+0000","last_change":"2024-04-22T07:04:03.793994+0000","last_active":"2024-04-22T07:04:04.860510+0000","last_peered":"2024-04-22T07:04:04.860510+0000","last_clean":"2024-04-22T07:04:04.860510+0000","last_became_active":"2024-04-22T07:04:03.793734+0000","last_became_peered":"2024-04-22T07:04:03.793734+0000","last_unstale":"2024-04-22T07:04:04.860510+0000","last_undegraded":"2024-04-22T07:04:04.860510+0000","last_fullsized":"2024-04-22T07:04:04.860510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860572+0000","last_change":"2024-04-22T07:04:02.791945+0000","last_active":"2024-04-22T07:04:04.860572+0000","last_peered":"2024-04-22T07:04:04.860572+0000","last_clean":"2024-04-22T07:04:04.860572+0000","last_became_active":"2024-04-22T07:04:02.791654+0000","last_became_peered":"2024-04-22T07:04:02.791654+0000","last_unstale":"2024-04-22T07:04:04.860572+0000","last_undegraded":"2024-04-22T07:04:04.860572+0000","last_fullsized":"2024-04-22T07:04:04.860572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860230+0000","last_change":"2024-04-22T07:04:03.797391+0000","last_active":"2024-04-22T07:04:04.860230+0000","last_peered":"2024-04-22T07:04:04.860230+0000","last_clean":"2024-04-22T07:04:04.860230+0000","last_became_active":"2024-04-22T07:04:03.797123+0000","last_became_peered":"2024-04-22T07:04:03.797123+0000","last_unstale":"2024-04-22T07:04:04.860230+0000","last_undegraded":"2024-04-22T07:04:04.860230+0000","last_fullsized":"2024-04-22T07:04:04.860230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.785515+0000","last_change":"2024-04-22T07:04:02.793753+0000","last_active":"2024-04-22T07:04:08.785515+0000","last_peered":"2024-04-22T07:04:08.785515+0000","last_clean":"2024-04-22T07:04:08.785515+0000","last_became_active":"2024-04-22T07:04:02.793436+0000","last_became_peered":"2024-04-22T07:04:02.793436+0000","last_unstale":"2024-04-22T07:04:08.785515+0000","last_undegraded":"2024-04-22T07:04:08.785515+0000","last_fullsized":"2024-04-22T07:04:08.785515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790527+0000","last_change":"2024-04-22T07:04:03.800665+0000","last_active":"2024-04-22T07:04:04.790527+0000","last_peered":"2024-04-22T07:04:04.790527+0000","last_clean":"2024-04-22T07:04:04.790527+0000","last_became_active":"2024-04-22T07:04:03.800227+0000","last_became_peered":"2024-04-22T07:04:03.800227+0000","last_unstale":"2024-04-22T07:04:04.790527+0000","last_undegraded":"2024-04-22T07:04:04.790527+0000","last_fullsized":"2024-04-22T07:04:04.790527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.784027+0000","last_change":"2024-04-22T07:04:02.791309+0000","last_active":"2024-04-22T07:04:08.784027+0000","last_peered":"2024-04-22T07:04:08.784027+0000","last_clean":"2024-04-22T07:04:08.784027+0000","last_became_active":"2024-04-22T07:04:02.791000+0000","last_became_peered":"2024-04-22T07:04:02.791000+0000","last_unstale":"2024-04-22T07:04:08.784027+0000","last_undegraded":"2024-04-22T07:04:08.784027+0000","last_fullsized":"2024-04-22T07:04:08.784027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861543+0000","last_change":"2024-04-22T07:04:03.794316+0000","last_active":"2024-04-22T07:04:04.861543+0000","last_peered":"2024-04-22T07:04:04.861543+0000","last_clean":"2024-04-22T07:04:04.861543+0000","last_became_active":"2024-04-22T07:04:03.793646+0000","last_became_peered":"2024-04-22T07:04:03.793646+0000","last_unstale":"2024-04-22T07:04:04.861543+0000","last_undegraded":"2024-04-22T07:04:04.861543+0000","last_fullsized":"2024-04-22T07:04:04.861543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.782279+0000","last_change":"2024-04-22T07:04:02.791024+0000","last_active":"2024-04-22T07:04:08.782279+0000","last_peered":"2024-04-22T07:04:08.782279+0000","last_clean":"2024-04-22T07:04:08.782279+0000","last_became_active":"2024-04-22T07:04:02.789160+0000","last_became_peered":"2024-04-22T07:04:02.789160+0000","last_unstale":"2024-04-22T07:04:08.782279+0000","last_undegraded":"2024-04-22T07:04:08.782279+0000","last_fullsized":"2024-04-22T07:04:08.782279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859792+0000","last_change":"2024-04-22T07:04:03.798436+0000","last_active":"2024-04-22T07:04:04.859792+0000","last_peered":"2024-04-22T07:04:04.859792+0000","last_clean":"2024-04-22T07:04:04.859792+0000","last_became_active":"2024-04-22T07:04:03.797946+0000","last_became_peered":"2024-04-22T07:04:03.797946+0000","last_unstale":"2024-04-22T07:04:04.859792+0000","last_undegraded":"2024-04-22T07:04:04.859792+0000","last_fullsized":"2024-04-22T07:04:04.859792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861234+0000","last_change":"2024-04-22T07:04:02.796551+0000","last_active":"2024-04-22T07:04:04.861234+0000","last_peered":"2024-04-22T07:04:04.861234+0000","last_clean":"2024-04-22T07:04:04.861234+0000","last_became_active":"2024-04-22T07:04:02.796332+0000","last_became_peered":"2024-04-22T07:04:02.796332+0000","last_unstale":"2024-04-22T07:04:04.861234+0000","last_undegraded":"2024-04-22T07:04:04.861234+0000","last_fullsized":"2024-04-22T07:04:04.861234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:15.854295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861758+0000","last_change":"2024-04-22T07:04:03.804622+0000","last_active":"2024-04-22T07:04:04.861758+0000","last_peered":"2024-04-22T07:04:04.861758+0000","last_clean":"2024-04-22T07:04:04.861758+0000","last_became_active":"2024-04-22T07:04:03.804324+0000","last_became_peered":"2024-04-22T07:04:03.804324+0000","last_unstale":"2024-04-22T07:04:04.861758+0000","last_undegraded":"2024-04-22T07:04:04.861758+0000","last_fullsized":"2024-04-22T07:04:04.861758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860167+0000","last_change":"2024-04-22T07:04:02.787085+0000","last_active":"2024-04-22T07:04:04.860167+0000","last_peered":"2024-04-22T07:04:04.860167+0000","last_clean":"2024-04-22T07:04:04.860167+0000","last_became_active":"2024-04-22T07:04:02.786777+0000","last_became_peered":"2024-04-22T07:04:02.786777+0000","last_unstale":"2024-04-22T07:04:04.860167+0000","last_undegraded":"2024-04-22T07:04:04.860167+0000","last_fullsized":"2024-04-22T07:04:04.860167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860535+0000","last_change":"2024-04-22T07:04:03.796356+0000","last_active":"2024-04-22T07:04:04.860535+0000","last_peered":"2024-04-22T07:04:04.860535+0000","last_clean":"2024-04-22T07:04:04.860535+0000","last_became_active":"2024-04-22T07:04:03.796030+0000","last_became_peered":"2024-04-22T07:04:03.796030+0000","last_unstale":"2024-04-22T07:04:04.860535+0000","last_undegraded":"2024-04-22T07:04:04.860535+0000","last_fullsized":"2024-04-22T07:04:04.860535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.786454+0000","last_change":"2024-04-22T07:04:02.787425+0000","last_active":"2024-04-22T07:04:08.786454+0000","last_peered":"2024-04-22T07:04:08.786454+0000","last_clean":"2024-04-22T07:04:08.786454+0000","last_became_active":"2024-04-22T07:04:02.787146+0000","last_became_peered":"2024-04-22T07:04:02.787146+0000","last_unstale":"2024-04-22T07:04:08.786454+0000","last_undegraded":"2024-04-22T07:04:08.786454+0000","last_fullsized":"2024-04-22T07:04:08.786454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861631+0000","last_change":"2024-04-22T07:04:03.802924+0000","last_active":"2024-04-22T07:04:04.861631+0000","last_peered":"2024-04-22T07:04:04.861631+0000","last_clean":"2024-04-22T07:04:04.861631+0000","last_became_active":"2024-04-22T07:04:03.802644+0000","last_became_peered":"2024-04-22T07:04:03.802644+0000","last_unstale":"2024-04-22T07:04:04.861631+0000","last_undegraded":"2024-04-22T07:04:04.861631+0000","last_fullsized":"2024-04-22T07:04:04.861631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790421+0000","last_change":"2024-04-22T07:04:02.794481+0000","last_active":"2024-04-22T07:04:04.790421+0000","last_peered":"2024-04-22T07:04:04.790421+0000","last_clean":"2024-04-22T07:04:04.790421+0000","last_became_active":"2024-04-22T07:04:02.793982+0000","last_became_peered":"2024-04-22T07:04:02.793982+0000","last_unstale":"2024-04-22T07:04:04.790421+0000","last_undegraded":"2024-04-22T07:04:04.790421+0000","last_fullsized":"2024-04-22T07:04:04.790421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:58.906416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862139+0000","last_change":"2024-04-22T07:04:03.802724+0000","last_active":"2024-04-22T07:04:04.862139+0000","last_peered":"2024-04-22T07:04:04.862139+0000","last_clean":"2024-04-22T07:04:04.862139+0000","last_became_active":"2024-04-22T07:04:03.802434+0000","last_became_peered":"2024-04-22T07:04:03.802434+0000","last_unstale":"2024-04-22T07:04:04.862139+0000","last_undegraded":"2024-04-22T07:04:04.862139+0000","last_fullsized":"2024-04-22T07:04:04.862139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.781655+0000","last_change":"2024-04-22T07:04:02.791375+0000","last_active":"2024-04-22T07:04:08.781655+0000","last_peered":"2024-04-22T07:04:08.781655+0000","last_clean":"2024-04-22T07:04:08.781655+0000","last_became_active":"2024-04-22T07:04:02.791095+0000","last_became_peered":"2024-04-22T07:04:02.791095+0000","last_unstale":"2024-04-22T07:04:08.781655+0000","last_undegraded":"2024-04-22T07:04:08.781655+0000","last_fullsized":"2024-04-22T07:04:08.781655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862265+0000","last_change":"2024-04-22T07:04:03.802028+0000","last_active":"2024-04-22T07:04:04.862265+0000","last_peered":"2024-04-22T07:04:04.862265+0000","last_clean":"2024-04-22T07:04:04.862265+0000","last_became_active":"2024-04-22T07:04:03.801715+0000","last_became_peered":"2024-04-22T07:04:03.801715+0000","last_unstale":"2024-04-22T07:04:04.862265+0000","last_undegraded":"2024-04-22T07:04:04.862265+0000","last_fullsized":"2024-04-22T07:04:04.862265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:29.815068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788145+0000","last_change":"2024-04-22T07:04:02.793270+0000","last_active":"2024-04-22T07:04:04.788145+0000","last_peered":"2024-04-22T07:04:04.788145+0000","last_clean":"2024-04-22T07:04:04.788145+0000","last_became_active":"2024-04-22T07:04:02.793009+0000","last_became_peered":"2024-04-22T07:04:02.793009+0000","last_unstale":"2024-04-22T07:04:04.788145+0000","last_undegraded":"2024-04-22T07:04:04.788145+0000","last_fullsized":"2024-04-22T07:04:04.788145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861459+0000","last_change":"2024-04-22T07:04:03.800175+0000","last_active":"2024-04-22T07:04:04.861459+0000","last_peered":"2024-04-22T07:04:04.861459+0000","last_clean":"2024-04-22T07:04:04.861459+0000","last_became_active":"2024-04-22T07:04:03.799902+0000","last_became_peered":"2024-04-22T07:04:03.799902+0000","last_unstale":"2024-04-22T07:04:04.861459+0000","last_undegraded":"2024-04-22T07:04:04.861459+0000","last_fullsized":"2024-04-22T07:04:04.861459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859806+0000","last_change":"2024-04-22T07:04:02.793643+0000","last_active":"2024-04-22T07:04:04.859806+0000","last_peered":"2024-04-22T07:04:04.859806+0000","last_clean":"2024-04-22T07:04:04.859806+0000","last_became_active":"2024-04-22T07:04:02.793333+0000","last_became_peered":"2024-04-22T07:04:02.793333+0000","last_unstale":"2024-04-22T07:04:04.859806+0000","last_undegraded":"2024-04-22T07:04:04.859806+0000","last_fullsized":"2024-04-22T07:04:04.859806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862019+0000","last_change":"2024-04-22T07:04:03.801821+0000","last_active":"2024-04-22T07:04:04.862019+0000","last_peered":"2024-04-22T07:04:04.862019+0000","last_clean":"2024-04-22T07:04:04.862019+0000","last_became_active":"2024-04-22T07:04:03.801510+0000","last_became_peered":"2024-04-22T07:04:03.801510+0000","last_unstale":"2024-04-22T07:04:04.862019+0000","last_undegraded":"2024-04-22T07:04:04.862019+0000","last_fullsized":"2024-04-22T07:04:04.862019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861241+0000","last_change":"2024-04-22T07:03:59.215285+0000","last_active":"2024-04-22T07:04:04.861241+0000","last_peered":"2024-04-22T07:04:04.861241+0000","last_clean":"2024-04-22T07:04:04.861241+0000","last_became_active":"2024-04-22T07:03:59.214275+0000","last_became_peered":"2024-04-22T07:03:59.214275+0000","last_unstale":"2024-04-22T07:04:04.861241+0000","last_undegraded":"2024-04-22T07:04:04.861241+0000","last_fullsized":"2024-04-22T07:04:04.861241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_clean_scrub_stamp":"2024-04-22T07:03:57.959761+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:55.105073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860226+0000","last_change":"2024-04-22T07:04:02.793384+0000","last_active":"2024-04-22T07:04:04.860226+0000","last_peered":"2024-04-22T07:04:04.860226+0000","last_clean":"2024-04-22T07:04:04.860226+0000","last_became_active":"2024-04-22T07:04:02.793077+0000","last_became_peered":"2024-04-22T07:04:02.793077+0000","last_unstale":"2024-04-22T07:04:04.860226+0000","last_undegraded":"2024-04-22T07:04:04.860226+0000","last_fullsized":"2024-04-22T07:04:04.860226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860779+0000","last_change":"2024-04-22T07:04:03.798134+0000","last_active":"2024-04-22T07:04:04.860779+0000","last_peered":"2024-04-22T07:04:04.860779+0000","last_clean":"2024-04-22T07:04:04.860779+0000","last_became_active":"2024-04-22T07:04:03.797848+0000","last_became_peered":"2024-04-22T07:04:03.797848+0000","last_unstale":"2024-04-22T07:04:04.860779+0000","last_undegraded":"2024-04-22T07:04:04.860779+0000","last_fullsized":"2024-04-22T07:04:04.860779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859966+0000","last_change":"2024-04-22T07:04:02.789591+0000","last_active":"2024-04-22T07:04:04.859966+0000","last_peered":"2024-04-22T07:04:04.859966+0000","last_clean":"2024-04-22T07:04:04.859966+0000","last_became_active":"2024-04-22T07:04:02.788785+0000","last_became_peered":"2024-04-22T07:04:02.788785+0000","last_unstale":"2024-04-22T07:04:04.859966+0000","last_undegraded":"2024-04-22T07:04:04.859966+0000","last_fullsized":"2024-04-22T07:04:04.859966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859611+0000","last_change":"2024-04-22T07:04:03.789757+0000","last_active":"2024-04-22T07:04:04.859611+0000","last_peered":"2024-04-22T07:04:04.859611+0000","last_clean":"2024-04-22T07:04:04.859611+0000","last_became_active":"2024-04-22T07:04:03.789463+0000","last_became_peered":"2024-04-22T07:04:03.789463+0000","last_unstale":"2024-04-22T07:04:04.859611+0000","last_undegraded":"2024-04-22T07:04:04.859611+0000","last_fullsized":"2024-04-22T07:04:04.859611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:16.859332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859989+0000","last_change":"2024-04-22T07:04:02.790600+0000","last_active":"2024-04-22T07:04:04.859989+0000","last_peered":"2024-04-22T07:04:04.859989+0000","last_clean":"2024-04-22T07:04:04.859989+0000","last_became_active":"2024-04-22T07:04:02.790319+0000","last_became_peered":"2024-04-22T07:04:02.790319+0000","last_unstale":"2024-04-22T07:04:04.859989+0000","last_undegraded":"2024-04-22T07:04:04.859989+0000","last_fullsized":"2024-04-22T07:04:04.859989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787840+0000","last_change":"2024-04-22T07:04:03.788231+0000","last_active":"2024-04-22T07:04:04.787840+0000","last_peered":"2024-04-22T07:04:04.787840+0000","last_clean":"2024-04-22T07:04:04.787840+0000","last_became_active":"2024-04-22T07:04:03.787918+0000","last_became_peered":"2024-04-22T07:04:03.787918+0000","last_unstale":"2024-04-22T07:04:04.787840+0000","last_undegraded":"2024-04-22T07:04:04.787840+0000","last_fullsized":"2024-04-22T07:04:04.787840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861257+0000","last_change":"2024-04-22T07:04:02.796245+0000","last_active":"2024-04-22T07:04:04.861257+0000","last_peered":"2024-04-22T07:04:04.861257+0000","last_clean":"2024-04-22T07:04:04.861257+0000","last_became_active":"2024-04-22T07:04:02.795947+0000","last_became_peered":"2024-04-22T07:04:02.795947+0000","last_unstale":"2024-04-22T07:04:04.861257+0000","last_undegraded":"2024-04-22T07:04:04.861257+0000","last_fullsized":"2024-04-22T07:04:04.861257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790274+0000","last_change":"2024-04-22T07:04:03.793330+0000","last_active":"2024-04-22T07:04:04.790274+0000","last_peered":"2024-04-22T07:04:04.790274+0000","last_clean":"2024-04-22T07:04:04.790274+0000","last_became_active":"2024-04-22T07:04:03.793037+0000","last_became_peered":"2024-04-22T07:04:03.793037+0000","last_unstale":"2024-04-22T07:04:04.790274+0000","last_undegraded":"2024-04-22T07:04:04.790274+0000","last_fullsized":"2024-04-22T07:04:04.790274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:13.368853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787950+0000","last_change":"2024-04-22T07:04:02.793498+0000","last_active":"2024-04-22T07:04:04.787950+0000","last_peered":"2024-04-22T07:04:04.787950+0000","last_clean":"2024-04-22T07:04:04.787950+0000","last_became_active":"2024-04-22T07:04:02.793215+0000","last_became_peered":"2024-04-22T07:04:02.793215+0000","last_unstale":"2024-04-22T07:04:04.787950+0000","last_undegraded":"2024-04-22T07:04:04.787950+0000","last_fullsized":"2024-04-22T07:04:04.787950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860866+0000","last_change":"2024-04-22T07:04:03.804152+0000","last_active":"2024-04-22T07:04:04.860866+0000","last_peered":"2024-04-22T07:04:04.860866+0000","last_clean":"2024-04-22T07:04:04.860866+0000","last_became_active":"2024-04-22T07:04:03.801961+0000","last_became_peered":"2024-04-22T07:04:03.801961+0000","last_unstale":"2024-04-22T07:04:04.860866+0000","last_undegraded":"2024-04-22T07:04:04.860866+0000","last_fullsized":"2024-04-22T07:04:04.860866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862105+0000","last_change":"2024-04-22T07:04:02.793445+0000","last_active":"2024-04-22T07:04:04.862105+0000","last_peered":"2024-04-22T07:04:04.862105+0000","last_clean":"2024-04-22T07:04:04.862105+0000","last_became_active":"2024-04-22T07:04:02.793164+0000","last_became_peered":"2024-04-22T07:04:02.793164+0000","last_unstale":"2024-04-22T07:04:04.862105+0000","last_undegraded":"2024-04-22T07:04:04.862105+0000","last_fullsized":"2024-04-22T07:04:04.862105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788013+0000","last_change":"2024-04-22T07:04:03.789589+0000","last_active":"2024-04-22T07:04:04.788013+0000","last_peered":"2024-04-22T07:04:04.788013+0000","last_clean":"2024-04-22T07:04:04.788013+0000","last_became_active":"2024-04-22T07:04:03.789310+0000","last_became_peered":"2024-04-22T07:04:03.789310+0000","last_unstale":"2024-04-22T07:04:04.788013+0000","last_undegraded":"2024-04-22T07:04:04.788013+0000","last_fullsized":"2024-04-22T07:04:04.788013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788077+0000","last_change":"2024-04-22T07:04:02.789243+0000","last_active":"2024-04-22T07:04:04.788077+0000","last_peered":"2024-04-22T07:04:04.788077+0000","last_clean":"2024-04-22T07:04:04.788077+0000","last_became_active":"2024-04-22T07:04:02.788897+0000","last_became_peered":"2024-04-22T07:04:02.788897+0000","last_unstale":"2024-04-22T07:04:04.788077+0000","last_undegraded":"2024-04-22T07:04:04.788077+0000","last_fullsized":"2024-04-22T07:04:04.788077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860272+0000","last_change":"2024-04-22T07:04:03.793656+0000","last_active":"2024-04-22T07:04:04.860272+0000","last_peered":"2024-04-22T07:04:04.860272+0000","last_clean":"2024-04-22T07:04:04.860272+0000","last_became_active":"2024-04-22T07:04:03.793311+0000","last_became_peered":"2024-04-22T07:04:03.793311+0000","last_unstale":"2024-04-22T07:04:04.860272+0000","last_undegraded":"2024-04-22T07:04:04.860272+0000","last_fullsized":"2024-04-22T07:04:04.860272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.783618+0000","last_change":"2024-04-22T07:04:02.786878+0000","last_active":"2024-04-22T07:04:08.783618+0000","last_peered":"2024-04-22T07:04:08.783618+0000","last_clean":"2024-04-22T07:04:08.783618+0000","last_became_active":"2024-04-22T07:04:02.786526+0000","last_became_peered":"2024-04-22T07:04:02.786526+0000","last_unstale":"2024-04-22T07:04:08.783618+0000","last_undegraded":"2024-04-22T07:04:08.783618+0000","last_fullsized":"2024-04-22T07:04:08.783618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860520+0000","last_change":"2024-04-22T07:04:03.791544+0000","last_active":"2024-04-22T07:04:04.860520+0000","last_peered":"2024-04-22T07:04:04.860520+0000","last_clean":"2024-04-22T07:04:04.860520+0000","last_became_active":"2024-04-22T07:04:03.791289+0000","last_became_peered":"2024-04-22T07:04:03.791289+0000","last_unstale":"2024-04-22T07:04:04.860520+0000","last_undegraded":"2024-04-22T07:04:04.860520+0000","last_fullsized":"2024-04-22T07:04:04.860520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859963+0000","last_change":"2024-04-22T07:04:02.795178+0000","last_active":"2024-04-22T07:04:04.859963+0000","last_peered":"2024-04-22T07:04:04.859963+0000","last_clean":"2024-04-22T07:04:04.859963+0000","last_became_active":"2024-04-22T07:04:02.794795+0000","last_became_peered":"2024-04-22T07:04:02.794795+0000","last_unstale":"2024-04-22T07:04:04.859963+0000","last_undegraded":"2024-04-22T07:04:04.859963+0000","last_fullsized":"2024-04-22T07:04:04.859963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790163+0000","last_change":"2024-04-22T07:04:03.788560+0000","last_active":"2024-04-22T07:04:04.790163+0000","last_peered":"2024-04-22T07:04:04.790163+0000","last_clean":"2024-04-22T07:04:04.790163+0000","last_became_active":"2024-04-22T07:04:03.788278+0000","last_became_peered":"2024-04-22T07:04:03.788278+0000","last_unstale":"2024-04-22T07:04:04.790163+0000","last_undegraded":"2024-04-22T07:04:04.790163+0000","last_fullsized":"2024-04-22T07:04:04.790163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:37.492038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787876+0000","last_change":"2024-04-22T07:04:02.788532+0000","last_active":"2024-04-22T07:04:04.787876+0000","last_peered":"2024-04-22T07:04:04.787876+0000","last_clean":"2024-04-22T07:04:04.787876+0000","last_became_active":"2024-04-22T07:04:02.788254+0000","last_became_peered":"2024-04-22T07:04:02.788254+0000","last_unstale":"2024-04-22T07:04:04.787876+0000","last_undegraded":"2024-04-22T07:04:04.787876+0000","last_fullsized":"2024-04-22T07:04:04.787876+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859748+0000","last_change":"2024-04-22T07:04:03.790052+0000","last_active":"2024-04-22T07:04:04.859748+0000","last_peered":"2024-04-22T07:04:04.859748+0000","last_clean":"2024-04-22T07:04:04.859748+0000","last_became_active":"2024-04-22T07:04:03.789351+0000","last_became_peered":"2024-04-22T07:04:03.789351+0000","last_unstale":"2024-04-22T07:04:04.859748+0000","last_undegraded":"2024-04-22T07:04:04.859748+0000","last_fullsized":"2024-04-22T07:04:04.859748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787903+0000","last_change":"2024-04-22T07:04:02.792774+0000","last_active":"2024-04-22T07:04:04.787903+0000","last_peered":"2024-04-22T07:04:04.787903+0000","last_clean":"2024-04-22T07:04:04.787903+0000","last_became_active":"2024-04-22T07:04:02.792282+0000","last_became_peered":"2024-04-22T07:04:02.792282+0000","last_unstale":"2024-04-22T07:04:04.787903+0000","last_undegraded":"2024-04-22T07:04:04.787903+0000","last_fullsized":"2024-04-22T07:04:04.787903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860691+0000","last_change":"2024-04-22T07:04:03.796133+0000","last_active":"2024-04-22T07:04:04.860691+0000","last_peered":"2024-04-22T07:04:04.860691+0000","last_clean":"2024-04-22T07:04:04.860691+0000","last_became_active":"2024-04-22T07:04:03.795854+0000","last_became_peered":"2024-04-22T07:04:03.795854+0000","last_unstale":"2024-04-22T07:04:04.860691+0000","last_undegraded":"2024-04-22T07:04:04.860691+0000","last_fullsized":"2024-04-22T07:04:04.860691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.783241+0000","last_change":"2024-04-22T07:04:02.789099+0000","last_active":"2024-04-22T07:04:08.783241+0000","last_peered":"2024-04-22T07:04:08.783241+0000","last_clean":"2024-04-22T07:04:08.783241+0000","last_became_active":"2024-04-22T07:04:02.788782+0000","last_became_peered":"2024-04-22T07:04:02.788782+0000","last_unstale":"2024-04-22T07:04:08.783241+0000","last_undegraded":"2024-04-22T07:04:08.783241+0000","last_fullsized":"2024-04-22T07:04:08.783241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860112+0000","last_change":"2024-04-22T07:04:03.799845+0000","last_active":"2024-04-22T07:04:04.860112+0000","last_peered":"2024-04-22T07:04:04.860112+0000","last_clean":"2024-04-22T07:04:04.860112+0000","last_became_active":"2024-04-22T07:04:03.799553+0000","last_became_peered":"2024-04-22T07:04:03.799553+0000","last_unstale":"2024-04-22T07:04:04.860112+0000","last_undegraded":"2024-04-22T07:04:04.860112+0000","last_fullsized":"2024-04-22T07:04:04.860112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860699+0000","last_change":"2024-04-22T07:04:03.799918+0000","last_active":"2024-04-22T07:04:04.860699+0000","last_peered":"2024-04-22T07:04:04.860699+0000","last_clean":"2024-04-22T07:04:04.860699+0000","last_became_active":"2024-04-22T07:04:03.797851+0000","last_became_peered":"2024-04-22T07:04:03.797851+0000","last_unstale":"2024-04-22T07:04:04.860699+0000","last_undegraded":"2024-04-22T07:04:04.860699+0000","last_fullsized":"2024-04-22T07:04:04.860699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.782807+0000","last_change":"2024-04-22T07:04:02.793627+0000","last_active":"2024-04-22T07:04:08.782807+0000","last_peered":"2024-04-22T07:04:08.782807+0000","last_clean":"2024-04-22T07:04:08.782807+0000","last_became_active":"2024-04-22T07:04:02.793341+0000","last_became_peered":"2024-04-22T07:04:02.793341+0000","last_unstale":"2024-04-22T07:04:08.782807+0000","last_undegraded":"2024-04-22T07:04:08.782807+0000","last_fullsized":"2024-04-22T07:04:08.782807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:21:07.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861294+0000","last_change":"2024-04-22T07:04:03.797768+0000","last_active":"2024-04-22T07:04:04.861294+0000","last_peered":"2024-04-22T07:04:04.861294+0000","last_clean":"2024-04-22T07:04:04.861294+0000","last_became_active":"2024-04-22T07:04:03.797437+0000","last_became_peered":"2024-04-22T07:04:03.797437+0000","last_unstale":"2024-04-22T07:04:04.861294+0000","last_undegraded":"2024-04-22T07:04:04.861294+0000","last_fullsized":"2024-04-22T07:04:04.861294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.783111+0000","last_change":"2024-04-22T07:04:02.789875+0000","last_active":"2024-04-22T07:04:08.783111+0000","last_peered":"2024-04-22T07:04:08.783111+0000","last_clean":"2024-04-22T07:04:08.783111+0000","last_became_active":"2024-04-22T07:04:02.788989+0000","last_became_peered":"2024-04-22T07:04:02.788989+0000","last_unstale":"2024-04-22T07:04:08.783111+0000","last_undegraded":"2024-04-22T07:04:08.783111+0000","last_fullsized":"2024-04-22T07:04:08.783111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860096+0000","last_change":"2024-04-22T07:04:03.797671+0000","last_active":"2024-04-22T07:04:04.860096+0000","last_peered":"2024-04-22T07:04:04.860096+0000","last_clean":"2024-04-22T07:04:04.860096+0000","last_became_active":"2024-04-22T07:04:03.797399+0000","last_became_peered":"2024-04-22T07:04:03.797399+0000","last_unstale":"2024-04-22T07:04:04.860096+0000","last_undegraded":"2024-04-22T07:04:04.860096+0000","last_fullsized":"2024-04-22T07:04:04.860096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860445+0000","last_change":"2024-04-22T07:04:02.796141+0000","last_active":"2024-04-22T07:04:04.860445+0000","last_peered":"2024-04-22T07:04:04.860445+0000","last_clean":"2024-04-22T07:04:04.860445+0000","last_became_active":"2024-04-22T07:04:02.795821+0000","last_became_peered":"2024-04-22T07:04:02.795821+0000","last_unstale":"2024-04-22T07:04:04.860445+0000","last_undegraded":"2024-04-22T07:04:04.860445+0000","last_fullsized":"2024-04-22T07:04:04.860445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860371+0000","last_change":"2024-04-22T07:04:03.796729+0000","last_active":"2024-04-22T07:04:04.860371+0000","last_peered":"2024-04-22T07:04:04.860371+0000","last_clean":"2024-04-22T07:04:04.860371+0000","last_became_active":"2024-04-22T07:04:03.796486+0000","last_became_peered":"2024-04-22T07:04:03.796486+0000","last_unstale":"2024-04-22T07:04:04.860371+0000","last_undegraded":"2024-04-22T07:04:04.860371+0000","last_fullsized":"2024-04-22T07:04:04.860371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.783665+0000","last_change":"2024-04-22T07:04:02.793214+0000","last_active":"2024-04-22T07:04:08.783665+0000","last_peered":"2024-04-22T07:04:08.783665+0000","last_clean":"2024-04-22T07:04:08.783665+0000","last_became_active":"2024-04-22T07:04:02.792922+0000","last_became_peered":"2024-04-22T07:04:02.792922+0000","last_unstale":"2024-04-22T07:04:08.783665+0000","last_undegraded":"2024-04-22T07:04:08.783665+0000","last_fullsized":"2024-04-22T07:04:08.783665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:15.169 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-22T07:04:15.170 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-22T07:04:15.170 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-22T07:04:15.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T07:04:15.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.304+0000 7f2664b9f640 1 Processor -- start 2024-04-22T07:04:15.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.304+0000 7f2664b9f640 1 -- start start 2024-04-22T07:04:15.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.305+0000 7f2664b9f640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f26600c8540 0x7f266015e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.305+0000 7f2664b9f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26600cd230 0x7f26600cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.305+0000 7f2664b9f640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26600c8990 con 0x7f2660154dc0 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.305+0000 7f2664b9f640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f266006ccb0 con 0x7f26600c8540 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.305+0000 7f2664b9f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f26600c8c60 con 0x7f26600cd230 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.305+0000 7f265ffff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26600cd230 0x7f26600cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.305+0000 7f2656ffd640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f26600c8540 0x7f266015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.305+0000 7f2656ffd640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f26600c8540 0x7f266015e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:54160/0 (socket says 172.21.15.92:54160) 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.305+0000 7f265ffff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26600cd230 0x7f26600cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:52746/0 (socket says 172.21.15.92:52746) 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.305+0000 7f2656ffd640 1 -- 172.21.15.92:0/3039893422 learned_addr learned my addr 172.21.15.92:0/3039893422 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.305+0000 7f265f7fe640 1 --1- 172.21.15.92:0/3039893422 >> v1:172.21.15.145:6789/0 conn(0x7f2660154dc0 0x7f26600c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:55276/0 (socket says 172.21.15.92:55276) 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.306+0000 7f2656ffd640 1 -- 172.21.15.92:0/3039893422 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26600cd230 msgr2=0x7f26600cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.306+0000 7f2656ffd640 1 --2- 172.21.15.92:0/3039893422 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26600cd230 0x7f26600cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.306+0000 7f2656ffd640 1 -- 172.21.15.92:0/3039893422 >> v1:172.21.15.145:6789/0 conn(0x7f2660154dc0 legacy=0x7f26600c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.306+0000 7f2656ffd640 1 -- 172.21.15.92:0/3039893422 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26600c8f40 con 0x7f26600c8540 2024-04-22T07:04:15.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.307+0000 7f2656ffd640 1 --2- 172.21.15.92:0/3039893422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26600c8540 0x7f266015e000 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2648009870 tx=0x7f264802ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da9614bd7c0267fb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:15.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.307+0000 7f265effd640 1 -- 172.21.15.92:0/3039893422 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264802f840 con 0x7f26600c8540 2024-04-22T07:04:15.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.307+0000 7f265effd640 1 -- 172.21.15.92:0/3039893422 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264802f9e0 con 0x7f26600c8540 2024-04-22T07:04:15.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.307+0000 7f265effd640 1 -- 172.21.15.92:0/3039893422 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264802fdb0 con 0x7f26600c8540 2024-04-22T07:04:15.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.307+0000 7f2664b9f640 1 -- 172.21.15.92:0/3039893422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26600c8540 msgr2=0x7f266015e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.307+0000 7f2664b9f640 1 --2- 172.21.15.92:0/3039893422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26600c8540 0x7f266015e000 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2648009870 tx=0x7f264802ba20 comp rx=0 tx=0).stop 2024-04-22T07:04:15.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.307+0000 7f2664b9f640 1 -- 172.21.15.92:0/3039893422 shutdown_connections 2024-04-22T07:04:15.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.307+0000 7f2664b9f640 1 --2- 172.21.15.92:0/3039893422 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26600cd230 0x7f26600cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.307+0000 7f2664b9f640 1 --2- 172.21.15.92:0/3039893422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26600c8540 0x7f266015e000 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.307+0000 7f2664b9f640 1 -- 172.21.15.92:0/3039893422 >> 172.21.15.92:0/3039893422 conn(0x7f266006e940 msgr2=0x7f266006ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:15.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.307+0000 7f2664b9f640 1 -- 172.21.15.92:0/3039893422 shutdown_connections 2024-04-22T07:04:15.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.307+0000 7f2664b9f640 1 -- 172.21.15.92:0/3039893422 wait complete. 2024-04-22T07:04:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.308+0000 7f2664b9f640 1 Processor -- start 2024-04-22T07:04:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.309+0000 7f2664b9f640 1 -- start start 2024-04-22T07:04:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.309+0000 7f2664b9f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26600c8540 0x7f2660158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.309+0000 7f2664b9f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26600cd230 0x7f266015a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.309+0000 7f2664b9f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2660154dc0 0x7f266015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.309+0000 7f2664b9f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f26601551a0 con 0x7f2660154dc0 2024-04-22T07:04:15.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.309+0000 7f2664b9f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f266006c940 con 0x7f26600c8540 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.309+0000 7f2664b9f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f26600c7280 con 0x7f26600cd230 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.309+0000 7f265ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2660154dc0 0x7f266015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.309+0000 7f265ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2660154dc0 0x7f266015a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36526/0 (socket says 172.21.15.92:36526) 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.309+0000 7f265ffff640 1 -- 172.21.15.92:0/1029046147 learned_addr learned my addr 172.21.15.92:0/1029046147 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f265f7fe640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26600c8540 0x7f2660158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f2656ffd640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26600cd230 0x7f266015a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f265ffff640 1 -- 172.21.15.92:0/1029046147 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26600cd230 msgr2=0x7f266015a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f265ffff640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26600cd230 0x7f266015a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f265ffff640 1 -- 172.21.15.92:0/1029046147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26600c8540 msgr2=0x7f2660158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f265ffff640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26600c8540 0x7f2660158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f265ffff640 1 -- 172.21.15.92:0/1029046147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26601f0ba0 con 0x7f2660154dc0 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f265f7fe640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26600c8540 0x7f2660158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f265ffff640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2660154dc0 0x7f266015a600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f265800efc0 tx=0x7f265800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:15.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f265cff9640 1 -- 172.21.15.92:0/1029046147 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658018020 con 0x7f2660154dc0 2024-04-22T07:04:15.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f265cff9640 1 -- 172.21.15.92:0/1029046147 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2658014480 con 0x7f2660154dc0 2024-04-22T07:04:15.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f2664b9f640 1 -- 172.21.15.92:0/1029046147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26601f0e80 con 0x7f2660154dc0 2024-04-22T07:04:15.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f265cff9640 1 -- 172.21.15.92:0/1029046147 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658014870 con 0x7f2660154dc0 2024-04-22T07:04:15.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.310+0000 7f2664b9f640 1 -- 172.21.15.92:0/1029046147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26601f1440 con 0x7f2660154dc0 2024-04-22T07:04:15.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.311+0000 7f2664b9f640 1 -- 172.21.15.92:0/1029046147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266004fd50 con 0x7f2660154dc0 2024-04-22T07:04:15.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.312+0000 7f265cff9640 1 -- 172.21.15.92:0/1029046147 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2658005d10 con 0x7f2660154dc0 2024-04-22T07:04:15.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.312+0000 7f265cff9640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f264006c8f0 0x7f264006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.312+0000 7f265f7fe640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f264006c8f0 0x7f264006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.312+0000 7f265cff9640 1 -- 172.21.15.92:0/1029046147 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f265808ca00 con 0x7f2660154dc0 2024-04-22T07:04:15.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.313+0000 7f265f7fe640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f264006c8f0 0x7f264006edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2650007fd0 tx=0x7f2650007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.317+0000 7f265cff9640 1 -- 172.21.15.92:0/1029046147 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2658022030 con 0x7f2660154dc0 2024-04-22T07:04:15.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.535+0000 7f2664b9f640 1 -- 172.21.15.92:0/1029046147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f266006c020 con 0x7f2660154dc0 2024-04-22T07:04:15.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.536+0000 7f265cff9640 1 -- 172.21.15.92:0/1029046147 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f2658010040 con 0x7f2660154dc0 2024-04-22T07:04:15.534 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:15.534 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-22T07:04:15.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.538+0000 7f2664b9f640 1 -- 172.21.15.92:0/1029046147 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f264006c8f0 msgr2=0x7f264006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.538+0000 7f2664b9f640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f264006c8f0 0x7f264006edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2650007fd0 tx=0x7f2650007d60 comp rx=0 tx=0).stop 2024-04-22T07:04:15.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.538+0000 7f2664b9f640 1 -- 172.21.15.92:0/1029046147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2660154dc0 msgr2=0x7f266015a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.538+0000 7f2664b9f640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2660154dc0 0x7f266015a600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f265800efc0 tx=0x7f265800c490 comp rx=0 tx=0).stop 2024-04-22T07:04:15.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.538+0000 7f2664b9f640 1 -- 172.21.15.92:0/1029046147 shutdown_connections 2024-04-22T07:04:15.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.538+0000 7f2664b9f640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f264006c8f0 0x7f264006edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.538+0000 7f2664b9f640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2660154dc0 0x7f266015a600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.538+0000 7f2664b9f640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26600cd230 0x7f266015a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.538+0000 7f2664b9f640 1 --2- 172.21.15.92:0/1029046147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26600c8540 0x7f2660158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.538+0000 7f2664b9f640 1 -- 172.21.15.92:0/1029046147 >> 172.21.15.92:0/1029046147 conn(0x7f266006e940 msgr2=0x7f26600cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:15.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.538+0000 7f2664b9f640 1 -- 172.21.15.92:0/1029046147 shutdown_connections 2024-04-22T07:04:15.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:15.538+0000 7f2664b9f640 1 -- 172.21.15.92:0/1029046147 wait complete. 2024-04-22T07:04:17.912 INFO:tasks.ceph.mds.e.smithi092.stderr:2024-04-22T07:04:17.914+0000 7ff94f4d7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-22T07:04:17.912 INFO:tasks.ceph.mds.e.smithi092.stderr:2024-04-22T07:04:17.914+0000 7ff94f4d7640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-22T07:04:18.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T07:04:18.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.645+0000 7fa8cdc43640 1 Processor -- start 2024-04-22T07:04:18.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.645+0000 7fa8cdc43640 1 -- start start 2024-04-22T07:04:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.646+0000 7fa8cdc43640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa8c8155040 0x7fa8c80c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.646+0000 7fa8cdc43640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8c815b320 0x7fa8c815b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.646+0000 7fa8cdc43640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c80739a0 con 0x7fa8c8073de0 2024-04-22T07:04:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.646+0000 7fa8cdc43640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa8c815bcb0 con 0x7fa8c8155040 2024-04-22T07:04:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.646+0000 7fa8cdc43640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c8155420 con 0x7fa8c815b320 2024-04-22T07:04:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.647+0000 7fa8c7fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8c815b320 0x7fa8c815b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.647+0000 7fa8c77fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa8c8155040 0x7fa8c80c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.647+0000 7fa8c77fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa8c8155040 0x7fa8c80c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:42726/0 (socket says 172.21.15.92:42726) 2024-04-22T07:04:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.647+0000 7fa8c77fe640 1 -- 172.21.15.92:0/903438444 learned_addr learned my addr 172.21.15.92:0/903438444 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.647+0000 7fa8beffd640 1 --1- 172.21.15.92:0/903438444 >> v1:172.21.15.145:6789/0 conn(0x7fa8c8073de0 0x7fa8c80c7fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:38668/0 (socket says 172.21.15.92:38668) 2024-04-22T07:04:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.647+0000 7fa8c77fe640 1 -- 172.21.15.92:0/903438444 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8c815b320 msgr2=0x7fa8c815b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.647+0000 7fa8c77fe640 1 --2- 172.21.15.92:0/903438444 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8c815b320 0x7fa8c815b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.647+0000 7fa8c77fe640 1 -- 172.21.15.92:0/903438444 >> v1:172.21.15.145:6789/0 conn(0x7fa8c8073de0 legacy=0x7fa8c80c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.647+0000 7fa8c6ffd640 1 -- 172.21.15.92:0/903438444 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1545852156 0 0) 0x7fa8c80739a0 con 0x7fa8c8073de0 2024-04-22T07:04:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.647+0000 7fa8c77fe640 1 -- 172.21.15.92:0/903438444 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c8072e20 con 0x7fa8c8155040 2024-04-22T07:04:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.647+0000 7fa8c7fff640 1 --2- 172.21.15.92:0/903438444 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8c815b320 0x7fa8c815b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.648+0000 7fa8c77fe640 1 --2- 172.21.15.92:0/903438444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c8155040 0x7fa8c80c7580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009870 tx=0x7fa8b802ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f6d2ec1f47f3927 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.648+0000 7fa8c6ffd640 1 -- 172.21.15.92:0/903438444 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b802f840 con 0x7fa8c8155040 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.648+0000 7fa8c6ffd640 1 -- 172.21.15.92:0/903438444 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b802f9e0 con 0x7fa8c8155040 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.648+0000 7fa8c6ffd640 1 -- 172.21.15.92:0/903438444 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b802fdb0 con 0x7fa8c8155040 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.648+0000 7fa8cdc43640 1 -- 172.21.15.92:0/903438444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c8155040 msgr2=0x7fa8c80c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.648+0000 7fa8cdc43640 1 --2- 172.21.15.92:0/903438444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c8155040 0x7fa8c80c7580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009870 tx=0x7fa8b802ba20 comp rx=0 tx=0).stop 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.648+0000 7fa8cdc43640 1 -- 172.21.15.92:0/903438444 shutdown_connections 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.648+0000 7fa8cdc43640 1 --2- 172.21.15.92:0/903438444 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8c815b320 0x7fa8c815b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.648+0000 7fa8cdc43640 1 --2- 172.21.15.92:0/903438444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c8155040 0x7fa8c80c7580 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.649+0000 7fa8cdc43640 1 -- 172.21.15.92:0/903438444 >> 172.21.15.92:0/903438444 conn(0x7fa8c80ccb80 msgr2=0x7fa8c80ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.649+0000 7fa8cdc43640 1 -- 172.21.15.92:0/903438444 shutdown_connections 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.649+0000 7fa8cdc43640 1 -- 172.21.15.92:0/903438444 wait complete. 2024-04-22T07:04:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.650+0000 7fa8cdc43640 1 Processor -- start 2024-04-22T07:04:18.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.650+0000 7fa8cdc43640 1 -- start start 2024-04-22T07:04:18.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.650+0000 7fa8cdc43640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c8073de0 0x7fa8c80c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.650+0000 7fa8cdc43640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c8155040 0x7fa8c81f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.650+0000 7fa8cdc43640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8c815b320 0x7fa8c814e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.650+0000 7fa8cdc43640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c80736c0 con 0x7fa8c8073de0 2024-04-22T07:04:18.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.650+0000 7fa8cdc43640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c80733e0 con 0x7fa8c8155040 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.650+0000 7fa8cdc43640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c815ae00 con 0x7fa8c815b320 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c8073de0 0x7fa8c80c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c8073de0 0x7fa8c80c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47244/0 (socket says 172.21.15.92:47244) 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c77fe640 1 -- 172.21.15.92:0/1404790125 learned_addr learned my addr 172.21.15.92:0/1404790125 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8beffd640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c8155040 0x7fa8c81f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c7fff640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8c815b320 0x7fa8c814e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c77fe640 1 -- 172.21.15.92:0/1404790125 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8c815b320 msgr2=0x7fa8c814e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c77fe640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8c815b320 0x7fa8c814e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c77fe640 1 -- 172.21.15.92:0/1404790125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c8155040 msgr2=0x7fa8c81f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c77fe640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c8155040 0x7fa8c81f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c77fe640 1 -- 172.21.15.92:0/1404790125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c81f0dc0 con 0x7fa8c8073de0 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c7fff640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8c815b320 0x7fa8c814e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8beffd640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c8155040 0x7fa8c81f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c77fe640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c8073de0 0x7fa8c80c6ad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8000c00 tx=0x7fa8b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:18.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c4ff9640 1 -- 172.21.15.92:0/1404790125 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8047070 con 0x7fa8c8073de0 2024-04-22T07:04:18.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c4ff9640 1 -- 172.21.15.92:0/1404790125 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b8004230 con 0x7fa8c8073de0 2024-04-22T07:04:18.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8cdc43640 1 -- 172.21.15.92:0/1404790125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c81f10a0 con 0x7fa8c8073de0 2024-04-22T07:04:18.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8c4ff9640 1 -- 172.21.15.92:0/1404790125 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8037940 con 0x7fa8c8073de0 2024-04-22T07:04:18.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.651+0000 7fa8cdc43640 1 -- 172.21.15.92:0/1404790125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c81f1660 con 0x7fa8c8073de0 2024-04-22T07:04:18.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.652+0000 7fa8cdc43640 1 -- 172.21.15.92:0/1404790125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c8063140 con 0x7fa8c8073de0 2024-04-22T07:04:18.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.653+0000 7fa8c4ff9640 1 -- 172.21.15.92:0/1404790125 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8b804a070 con 0x7fa8c8073de0 2024-04-22T07:04:18.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.653+0000 7fa8c4ff9640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa8a806c920 0x7fa8a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.653+0000 7fa8c4ff9640 1 -- 172.21.15.92:0/1404790125 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa8b80adb50 con 0x7fa8c8073de0 2024-04-22T07:04:18.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.653+0000 7fa8beffd640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa8a806c920 0x7fa8a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.654+0000 7fa8beffd640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa8a806c920 0x7fa8a806ede0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8c815ebb0 tx=0x7fa8b0007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:18.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.658+0000 7fa8c4ff9640 1 -- 172.21.15.92:0/1404790125 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa8c8063140 con 0x7fa8c8073de0 2024-04-22T07:04:18.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.879+0000 7fa8cdc43640 1 -- 172.21.15.92:0/1404790125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa8c8068b00 con 0x7fa8c8073de0 2024-04-22T07:04:18.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.879+0000 7fa8c4ff9640 1 -- 172.21.15.92:0/1404790125 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7fa8b8033030 con 0x7fa8c8073de0 2024-04-22T07:04:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:18.877 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-22T07:04:18.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.881+0000 7fa8cdc43640 1 -- 172.21.15.92:0/1404790125 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa8a806c920 msgr2=0x7fa8a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:18.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.881+0000 7fa8cdc43640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa8a806c920 0x7fa8a806ede0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8c815ebb0 tx=0x7fa8b0007c10 comp rx=0 tx=0).stop 2024-04-22T07:04:18.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.881+0000 7fa8cdc43640 1 -- 172.21.15.92:0/1404790125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c8073de0 msgr2=0x7fa8c80c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:18.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.881+0000 7fa8cdc43640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c8073de0 0x7fa8c80c6ad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8000c00 tx=0x7fa8b8004730 comp rx=0 tx=0).stop 2024-04-22T07:04:18.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.881+0000 7fa8cdc43640 1 -- 172.21.15.92:0/1404790125 shutdown_connections 2024-04-22T07:04:18.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.881+0000 7fa8cdc43640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa8a806c920 0x7fa8a806ede0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.881+0000 7fa8cdc43640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8c815b320 0x7fa8c814e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.881+0000 7fa8cdc43640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c8155040 0x7fa8c81f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.881+0000 7fa8cdc43640 1 --2- 172.21.15.92:0/1404790125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c8073de0 0x7fa8c80c6ad0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.881+0000 7fa8cdc43640 1 -- 172.21.15.92:0/1404790125 >> 172.21.15.92:0/1404790125 conn(0x7fa8c80ccb80 msgr2=0x7fa8c80c9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:18.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.881+0000 7fa8cdc43640 1 -- 172.21.15.92:0/1404790125 shutdown_connections 2024-04-22T07:04:18.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:18.881+0000 7fa8cdc43640 1 -- 172.21.15.92:0/1404790125 wait complete. 2024-04-22T07:04:21.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T07:04:21.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.990+0000 7f771e68e640 1 Processor -- start 2024-04-22T07:04:21.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.990+0000 7f771e68e640 1 -- start start 2024-04-22T07:04:21.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.991+0000 7f771e68e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7718159280 0x7f7718164500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.991+0000 7f771e68e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7718067f10 con 0x7f7718152c90 2024-04-22T07:04:21.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.991+0000 7f771e68e640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77180c7e50 con 0x7f7718153070 2024-04-22T07:04:21.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.991+0000 7f771e68e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f771806bd90 con 0x7f7718159280 2024-04-22T07:04:21.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.991+0000 7f771cc2b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7718159280 0x7f7718164500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.991+0000 7f7717fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7718152c90 0x7f77180c6a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44968/0 (socket says 172.21.15.92:44968) 2024-04-22T07:04:21.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.991+0000 7f7717fff640 1 -- 172.21.15.92:0/2396902929 learned_addr learned my addr 172.21.15.92:0/2396902929 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:21.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.991+0000 7f771cc2b640 1 -- 172.21.15.92:0/2396902929 >> v1:172.21.15.145:6789/0 conn(0x7f7718153070 legacy=0x7f7718158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.991+0000 7f771cc2b640 1 -- 172.21.15.92:0/2396902929 >> v1:172.21.15.92:6789/0 conn(0x7f7718152c90 legacy=0x7f77180c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.991+0000 7f771cc2b640 1 -- 172.21.15.92:0/2396902929 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77180c8110 con 0x7f7718159280 2024-04-22T07:04:21.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.991+0000 7f7716ffd640 1 -- 172.21.15.92:0/2396902929 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2256528309 0 0) 0x7f7718067f10 con 0x7f7718152c90 2024-04-22T07:04:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.992+0000 7f771cc2b640 1 --2- 172.21.15.92:0/2396902929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718159280 0x7f7718164500 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7710009d90 tx=0x7f771002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07be7a6eb5f3416 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.992+0000 7f7716ffd640 1 -- 172.21.15.92:0/2396902929 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710039070 con 0x7f7718159280 2024-04-22T07:04:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.992+0000 7f7716ffd640 1 -- 172.21.15.92:0/2396902929 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f771002cd00 con 0x7f7718159280 2024-04-22T07:04:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.992+0000 7f7716ffd640 1 -- 172.21.15.92:0/2396902929 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710034660 con 0x7f7718159280 2024-04-22T07:04:21.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.992+0000 7f771e68e640 1 -- 172.21.15.92:0/2396902929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718159280 msgr2=0x7f7718164500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.992+0000 7f771e68e640 1 --2- 172.21.15.92:0/2396902929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718159280 0x7f7718164500 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7710009d90 tx=0x7f771002bf50 comp rx=0 tx=0).stop 2024-04-22T07:04:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.992+0000 7f771e68e640 1 -- 172.21.15.92:0/2396902929 shutdown_connections 2024-04-22T07:04:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.992+0000 7f771e68e640 1 --2- 172.21.15.92:0/2396902929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718159280 0x7f7718164500 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.992+0000 7f771e68e640 1 -- 172.21.15.92:0/2396902929 >> 172.21.15.92:0/2396902929 conn(0x7f7718072ee0 msgr2=0x7f771806e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.992+0000 7f771e68e640 1 -- 172.21.15.92:0/2396902929 shutdown_connections 2024-04-22T07:04:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.992+0000 7f771e68e640 1 -- 172.21.15.92:0/2396902929 wait complete. 2024-04-22T07:04:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.993+0000 7f771e68e640 1 Processor -- start 2024-04-22T07:04:21.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.994+0000 7f771e68e640 1 -- start start 2024-04-22T07:04:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.994+0000 7f771e68e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7718152c90 0x7f7718144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.994+0000 7f771e68e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7718153070 0x7f77180cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.994+0000 7f771e68e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718159280 0x7f77181ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.994+0000 7f771e68e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7718067830 con 0x7f7718159280 2024-04-22T07:04:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.994+0000 7f771e68e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77180676b0 con 0x7f7718153070 2024-04-22T07:04:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.994+0000 7f771e68e640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f77180c83f0 con 0x7f7718152c90 2024-04-22T07:04:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.994+0000 7f771cc2b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718159280 0x7f77181ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.994+0000 7f771cc2b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718159280 0x7f77181ee6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47258/0 (socket says 172.21.15.92:47258) 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f771cc2b640 1 -- 172.21.15.92:0/2078822535 learned_addr learned my addr 172.21.15.92:0/2078822535 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f7717fff640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7718152c90 0x7f7718144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f77177fe640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7718153070 0x7f77180cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f771cc2b640 1 -- 172.21.15.92:0/2078822535 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7718152c90 msgr2=0x7f7718144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f771cc2b640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7718152c90 0x7f7718144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f771cc2b640 1 -- 172.21.15.92:0/2078822535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7718153070 msgr2=0x7f77180cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f771cc2b640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7718153070 0x7f77180cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f771cc2b640 1 -- 172.21.15.92:0/2078822535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7718145c90 con 0x7f7718159280 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f771cc2b640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718159280 0x7f77181ee6e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7710009d90 tx=0x7f7710004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f7714ff9640 1 -- 172.21.15.92:0/2078822535 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710039050 con 0x7f7718159280 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f7714ff9640 1 -- 172.21.15.92:0/2078822535 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7710045070 con 0x7f7718159280 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f7714ff9640 1 -- 172.21.15.92:0/2078822535 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771003e850 con 0x7f7718159280 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f771e68e640 1 -- 172.21.15.92:0/2078822535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7718145f70 con 0x7f7718159280 2024-04-22T07:04:21.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.995+0000 7f771e68e640 1 -- 172.21.15.92:0/2078822535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7718146530 con 0x7f7718159280 2024-04-22T07:04:21.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.996+0000 7f771e68e640 1 -- 172.21.15.92:0/2078822535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7718067c50 con 0x7f7718159280 2024-04-22T07:04:21.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.997+0000 7f7714ff9640 1 -- 172.21.15.92:0/2078822535 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f771003e9f0 con 0x7f7718159280 2024-04-22T07:04:21.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.997+0000 7f7714ff9640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f770406c8f0 0x7f770406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.997+0000 7f7714ff9640 1 -- 172.21.15.92:0/2078822535 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f77100ae980 con 0x7f7718159280 2024-04-22T07:04:21.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.997+0000 7f7717fff640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f770406c8f0 0x7f770406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:21.997+0000 7f7717fff640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f770406c8f0 0x7f770406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7708000f60 tx=0x7f7708007b90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.001+0000 7f7714ff9640 1 -- 172.21.15.92:0/2078822535 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f771007ce70 con 0x7f7718159280 2024-04-22T07:04:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.221+0000 7f771e68e640 1 -- 172.21.15.92:0/2078822535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f77181598e0 con 0x7f7718159280 2024-04-22T07:04:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.221+0000 7f7714ff9640 1 -- 172.21.15.92:0/2078822535 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f771007c770 con 0x7f7718159280 2024-04-22T07:04:22.219 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:22.219 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-22T07:04:22.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.223+0000 7f771e68e640 1 -- 172.21.15.92:0/2078822535 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f770406c8f0 msgr2=0x7f770406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.223+0000 7f771e68e640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f770406c8f0 0x7f770406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7708000f60 tx=0x7f7708007b90 comp rx=0 tx=0).stop 2024-04-22T07:04:22.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.223+0000 7f771e68e640 1 -- 172.21.15.92:0/2078822535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718159280 msgr2=0x7f77181ee6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.223+0000 7f771e68e640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718159280 0x7f77181ee6e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7710009d90 tx=0x7f7710004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.223+0000 7f771e68e640 1 -- 172.21.15.92:0/2078822535 shutdown_connections 2024-04-22T07:04:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.223+0000 7f771e68e640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f770406c8f0 0x7f770406edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.223+0000 7f771e68e640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718159280 0x7f77181ee6e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.223+0000 7f771e68e640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7718153070 0x7f77180cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.223+0000 7f771e68e640 1 --2- 172.21.15.92:0/2078822535 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7718152c90 0x7f7718144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.223+0000 7f771e68e640 1 -- 172.21.15.92:0/2078822535 >> 172.21.15.92:0/2078822535 conn(0x7f7718072ee0 msgr2=0x7f77181504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.223+0000 7f771e68e640 1 -- 172.21.15.92:0/2078822535 shutdown_connections 2024-04-22T07:04:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:22.223+0000 7f771e68e640 1 -- 172.21.15.92:0/2078822535 wait complete. 2024-04-22T07:04:22.912 INFO:tasks.ceph.mds.e.smithi092.stderr:2024-04-22T07:04:22.914+0000 7ff94f4d7640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-22T07:04:25.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T07:04:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.331+0000 7f4e99611640 1 Processor -- start 2024-04-22T07:04:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.331+0000 7f4e99611640 1 -- start start 2024-04-22T07:04:25.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.331+0000 7f4e99611640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e94151140 0x7f4e94156f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.331+0000 7f4e99611640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e940c7350 0x7f4e9406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.331+0000 7f4e99611640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e940ca310 con 0x7f4e940c6ec0 2024-04-22T07:04:25.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.331+0000 7f4e99611640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f4e940cb200 con 0x7f4e94151140 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.331+0000 7f4e99611640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4e940ca5f0 con 0x7f4e940c7350 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.332+0000 7f4e92ffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e94151140 0x7f4e94156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.332+0000 7f4e92ffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e94151140 0x7f4e94156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47262/0 (socket says 172.21.15.92:47262) 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.332+0000 7f4e92ffd640 1 -- 172.21.15.92:0/753465279 learned_addr learned my addr 172.21.15.92:0/753465279 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.332+0000 7f4e937fe640 1 --2- 172.21.15.92:0/753465279 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e940c7350 0x7f4e9406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.332+0000 7f4e92ffd640 1 -- 172.21.15.92:0/753465279 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e940c7350 msgr2=0x7f4e9406faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.332+0000 7f4e92ffd640 1 --2- 172.21.15.92:0/753465279 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e940c7350 0x7f4e9406faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.332+0000 7f4e92ffd640 1 -- 172.21.15.92:0/753465279 >> v1:172.21.15.145:6789/0 conn(0x7f4e940c6ec0 legacy=0x7f4e9406f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.332+0000 7f4e92ffd640 1 -- 172.21.15.92:0/753465279 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e940ca8d0 con 0x7f4e94151140 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.332+0000 7f4e937fe640 1 --2- 172.21.15.92:0/753465279 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e940c7350 0x7f4e9406faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.333+0000 7f4e92ffd640 1 --2- 172.21.15.92:0/753465279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151140 0x7f4e94156f60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400bb00 tx=0x7f4e8402dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0d6a82776451f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.333+0000 7f4e91ffb640 1 -- 172.21.15.92:0/753465279 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e84034070 con 0x7f4e94151140 2024-04-22T07:04:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.333+0000 7f4e91ffb640 1 -- 172.21.15.92:0/753465279 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e8402ed60 con 0x7f4e94151140 2024-04-22T07:04:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.333+0000 7f4e91ffb640 1 -- 172.21.15.92:0/753465279 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e84038620 con 0x7f4e94151140 2024-04-22T07:04:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.333+0000 7f4e99611640 1 -- 172.21.15.92:0/753465279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151140 msgr2=0x7f4e94156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.333+0000 7f4e99611640 1 --2- 172.21.15.92:0/753465279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151140 0x7f4e94156f60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400bb00 tx=0x7f4e8402dfb0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.333+0000 7f4e99611640 1 -- 172.21.15.92:0/753465279 shutdown_connections 2024-04-22T07:04:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.333+0000 7f4e99611640 1 --2- 172.21.15.92:0/753465279 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e940c7350 0x7f4e9406faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.333+0000 7f4e99611640 1 --2- 172.21.15.92:0/753465279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151140 0x7f4e94156f60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.333+0000 7f4e99611640 1 -- 172.21.15.92:0/753465279 >> 172.21.15.92:0/753465279 conn(0x7f4e940cc220 msgr2=0x7f4e940cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.333+0000 7f4e99611640 1 -- 172.21.15.92:0/753465279 shutdown_connections 2024-04-22T07:04:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.333+0000 7f4e99611640 1 -- 172.21.15.92:0/753465279 wait complete. 2024-04-22T07:04:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.335+0000 7f4e99611640 1 Processor -- start 2024-04-22T07:04:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.335+0000 7f4e99611640 1 -- start start 2024-04-22T07:04:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.336+0000 7f4e99611640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e940c6ec0 0x7f4e941f1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.336+0000 7f4e99611640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e940c7350 0x7f4e940c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.336+0000 7f4e99611640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151140 0x7f4e940c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.336+0000 7f4e99611640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4e940ca020 con 0x7f4e94151140 2024-04-22T07:04:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.336+0000 7f4e99611640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4e940cae90 con 0x7f4e940c6ec0 2024-04-22T07:04:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.336+0000 7f4e99611640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4e9406ffe0 con 0x7f4e940c7350 2024-04-22T07:04:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.336+0000 7f4e937fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151140 0x7f4e940c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.336+0000 7f4e937fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151140 0x7f4e940c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47278/0 (socket says 172.21.15.92:47278) 2024-04-22T07:04:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.336+0000 7f4e937fe640 1 -- 172.21.15.92:0/1387386296 learned_addr learned my addr 172.21.15.92:0/1387386296 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e927fc640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e940c7350 0x7f4e940c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e92ffd640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e940c6ec0 0x7f4e941f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e937fe640 1 -- 172.21.15.92:0/1387386296 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e940c7350 msgr2=0x7f4e940c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e937fe640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e940c7350 0x7f4e940c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e937fe640 1 -- 172.21.15.92:0/1387386296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e940c6ec0 msgr2=0x7f4e941f1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e937fe640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e940c6ec0 0x7f4e941f1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e937fe640 1 -- 172.21.15.92:0/1387386296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e940c9800 con 0x7f4e94151140 2024-04-22T07:04:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e92ffd640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e940c6ec0 0x7f4e941f1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e937fe640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151140 0x7f4e940c49a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009fd0 tx=0x7f4e8c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e7b7fe640 1 -- 172.21.15.92:0/1387386296 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c01a070 con 0x7f4e94151140 2024-04-22T07:04:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e7b7fe640 1 -- 172.21.15.92:0/1387386296 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e8c003bb0 con 0x7f4e94151140 2024-04-22T07:04:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e99611640 1 -- 172.21.15.92:0/1387386296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e940c9ae0 con 0x7f4e94151140 2024-04-22T07:04:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e7b7fe640 1 -- 172.21.15.92:0/1387386296 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c004030 con 0x7f4e94151140 2024-04-22T07:04:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.337+0000 7f4e99611640 1 -- 172.21.15.92:0/1387386296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e940724f0 con 0x7f4e94151140 2024-04-22T07:04:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.338+0000 7f4e99611640 1 -- 172.21.15.92:0/1387386296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e94069930 con 0x7f4e94151140 2024-04-22T07:04:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.339+0000 7f4e7b7fe640 1 -- 172.21.15.92:0/1387386296 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e8c01d070 con 0x7f4e94151140 2024-04-22T07:04:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.339+0000 7f4e7b7fe640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4e8006c8f0 0x7f4e8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.339+0000 7f4e7b7fe640 1 -- 172.21.15.92:0/1387386296 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4e8c08be90 con 0x7f4e94151140 2024-04-22T07:04:25.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.339+0000 7f4e92ffd640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4e8006c8f0 0x7f4e8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.339+0000 7f4e92ffd640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4e8006c8f0 0x7f4e8006edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e84039000 tx=0x7f4e84034000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:25.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.343+0000 7f4e7b7fe640 1 -- 172.21.15.92:0/1387386296 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e8c010070 con 0x7f4e94151140 2024-04-22T07:04:25.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.561+0000 7f4e99611640 1 -- 172.21.15.92:0/1387386296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4e9406f910 con 0x7f4e94151140 2024-04-22T07:04:25.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.561+0000 7f4e7b7fe640 1 -- 172.21.15.92:0/1387386296 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f4e8c05a2c0 con 0x7f4e94151140 2024-04-22T07:04:25.560 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:25.561 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-22T07:04:25.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.563+0000 7f4e99611640 1 -- 172.21.15.92:0/1387386296 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4e8006c8f0 msgr2=0x7f4e8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.563+0000 7f4e99611640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4e8006c8f0 0x7f4e8006edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e84039000 tx=0x7f4e84034000 comp rx=0 tx=0).stop 2024-04-22T07:04:25.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.563+0000 7f4e99611640 1 -- 172.21.15.92:0/1387386296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151140 msgr2=0x7f4e940c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.563+0000 7f4e99611640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151140 0x7f4e940c49a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009fd0 tx=0x7f4e8c00d560 comp rx=0 tx=0).stop 2024-04-22T07:04:25.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.563+0000 7f4e99611640 1 -- 172.21.15.92:0/1387386296 shutdown_connections 2024-04-22T07:04:25.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.563+0000 7f4e99611640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4e8006c8f0 0x7f4e8006edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.563+0000 7f4e99611640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151140 0x7f4e940c49a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.563+0000 7f4e99611640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e940c7350 0x7f4e940c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.563+0000 7f4e99611640 1 --2- 172.21.15.92:0/1387386296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e940c6ec0 0x7f4e941f1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.563+0000 7f4e99611640 1 -- 172.21.15.92:0/1387386296 >> 172.21.15.92:0/1387386296 conn(0x7f4e940cc220 msgr2=0x7f4e941553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:25.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.563+0000 7f4e99611640 1 -- 172.21.15.92:0/1387386296 shutdown_connections 2024-04-22T07:04:25.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.563+0000 7f4e99611640 1 -- 172.21.15.92:0/1387386296 wait complete. 2024-04-22T07:04:25.573 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-22T07:04:25.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:25.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.671+0000 7f53be504640 1 Processor -- start 2024-04-22T07:04:25.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.671+0000 7f53be504640 1 -- start start 2024-04-22T07:04:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53be504640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53b81583e0 0x7f53b80b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53be504640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53b806d680 0x7f53b80700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53be504640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f53b8070e20 0x7f53b8070600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53be504640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f53b81587c0 con 0x7f53b81583e0 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53be504640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f53b80c9670 con 0x7f53b8070e20 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53be504640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f53b80c9950 con 0x7f53b806d680 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53b7fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53b81583e0 0x7f53b80b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53b7fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f53b81583e0 0x7f53b80b0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47294/0 (socket says 172.21.15.92:47294) 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53b7fff640 1 -- 172.21.15.92:0/3504359399 learned_addr learned my addr 172.21.15.92:0/3504359399 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53b77fe640 1 --2- 172.21.15.92:0/3504359399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53b806d680 0x7f53b80700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53bcaa1640 1 --2- 172.21.15.92:0/3504359399 >> v2:172.21.15.145:3300/0 conn(0x7f53b8070e20 0x7f53b8070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53b7fff640 1 -- 172.21.15.92:0/3504359399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53b806d680 msgr2=0x7f53b80700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53b7fff640 1 --2- 172.21.15.92:0/3504359399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53b806d680 0x7f53b80700c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53b7fff640 1 -- 172.21.15.92:0/3504359399 >> v2:172.21.15.145:3300/0 conn(0x7f53b8070e20 msgr2=0x7f53b8070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53b7fff640 1 --2- 172.21.15.92:0/3504359399 >> v2:172.21.15.145:3300/0 conn(0x7f53b8070e20 0x7f53b8070600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.672+0000 7f53b7fff640 1 -- 172.21.15.92:0/3504359399 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b80c9c30 con 0x7f53b81583e0 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.673+0000 7f53b7fff640 1 --2- 172.21.15.92:0/3504359399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53b81583e0 0x7f53b80b0e30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f53b815bd60 tx=0x7f53ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aac372e56051bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.673+0000 7f53b6ffd640 1 -- 172.21.15.92:0/3504359399 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac035020 con 0x7f53b81583e0 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.673+0000 7f53b6ffd640 1 -- 172.21.15.92:0/3504359399 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ac031870 con 0x7f53b81583e0 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.673+0000 7f53b6ffd640 1 -- 172.21.15.92:0/3504359399 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac031c40 con 0x7f53b81583e0 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.674+0000 7f53be504640 1 -- 172.21.15.92:0/3504359399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53b81583e0 msgr2=0x7f53b80b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.674+0000 7f53be504640 1 --2- 172.21.15.92:0/3504359399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53b81583e0 0x7f53b80b0e30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f53b815bd60 tx=0x7f53ac02b910 comp rx=0 tx=0).stop 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.674+0000 7f53be504640 1 -- 172.21.15.92:0/3504359399 shutdown_connections 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.674+0000 7f53be504640 1 --2- 172.21.15.92:0/3504359399 >> v2:172.21.15.145:3300/0 conn(0x7f53b8070e20 0x7f53b8070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.674+0000 7f53be504640 1 --2- 172.21.15.92:0/3504359399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53b806d680 0x7f53b80700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.674+0000 7f53be504640 1 --2- 172.21.15.92:0/3504359399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53b81583e0 0x7f53b80b0e30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.674+0000 7f53be504640 1 -- 172.21.15.92:0/3504359399 >> 172.21.15.92:0/3504359399 conn(0x7f53b806db80 msgr2=0x7f53b806df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:25.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.674+0000 7f53be504640 1 -- 172.21.15.92:0/3504359399 shutdown_connections 2024-04-22T07:04:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.674+0000 7f53be504640 1 -- 172.21.15.92:0/3504359399 wait complete. 2024-04-22T07:04:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.675+0000 7f53be504640 1 Processor -- start 2024-04-22T07:04:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.675+0000 7f53be504640 1 -- start start 2024-04-22T07:04:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53be504640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53b806d680 0x7f53b80b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53be504640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53b8070e20 0x7f53b80b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53be504640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53b81583e0 0x7f53b8151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53be504640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f53b806bfd0 con 0x7f53b806d680 2024-04-22T07:04:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53be504640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f53b806bcf0 con 0x7f53b81583e0 2024-04-22T07:04:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53be504640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f53b815af80 con 0x7f53b8070e20 2024-04-22T07:04:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53b806d680 0x7f53b80b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53b806d680 0x7f53b80b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47302/0 (socket says 172.21.15.92:47302) 2024-04-22T07:04:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b7fff640 1 -- 172.21.15.92:0/2603898611 learned_addr learned my addr 172.21.15.92:0/2603898611 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b77fe640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53b8070e20 0x7f53b80b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53bcaa1640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53b81583e0 0x7f53b8151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b7fff640 1 -- 172.21.15.92:0/2603898611 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53b8070e20 msgr2=0x7f53b80b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b7fff640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53b8070e20 0x7f53b80b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b7fff640 1 -- 172.21.15.92:0/2603898611 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53b81583e0 msgr2=0x7f53b8151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b7fff640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53b81583e0 0x7f53b8151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b7fff640 1 -- 172.21.15.92:0/2603898611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b8150b30 con 0x7f53b806d680 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b7fff640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53b806d680 0x7f53b80b2410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f53ac02b950 tx=0x7f53ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b4ff9640 1 -- 172.21.15.92:0/2603898611 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac040070 con 0x7f53b806d680 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b4ff9640 1 -- 172.21.15.92:0/2603898611 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ac03a420 con 0x7f53b806d680 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53be504640 1 -- 172.21.15.92:0/2603898611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b8150e10 con 0x7f53b806d680 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53b4ff9640 1 -- 172.21.15.92:0/2603898611 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac035040 con 0x7f53b806d680 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.676+0000 7f53be504640 1 -- 172.21.15.92:0/2603898611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b81513d0 con 0x7f53b806d680 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.677+0000 7f53be504640 1 -- 172.21.15.92:0/2603898611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b8068b00 con 0x7f53b806d680 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.678+0000 7f53b4ff9640 1 -- 172.21.15.92:0/2603898611 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53ac04b060 con 0x7f53b806d680 2024-04-22T07:04:25.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.678+0000 7f53b4ff9640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f53a406c8f0 0x7f53a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.678+0000 7f53b77fe640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f53a406c8f0 0x7f53a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.678+0000 7f53b4ff9640 1 -- 172.21.15.92:0/2603898611 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53ac0adb20 con 0x7f53b806d680 2024-04-22T07:04:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.678+0000 7f53b77fe640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f53a406c8f0 0x7f53a406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f53b81507c0 tx=0x7f53a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.683+0000 7f53b4ff9640 1 -- 172.21.15.92:0/2603898611 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f53ac07bf50 con 0x7f53b806d680 2024-04-22T07:04:25.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.866+0000 7f53be504640 1 -- 172.21.15.92:0/2603898611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53b80b0a00 con 0x7f53b806d680 2024-04-22T07:04:25.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.867+0000 7f53b4ff9640 1 -- 172.21.15.92:0/2603898611 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f53ac07b850 con 0x7f53b806d680 2024-04-22T07:04:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:25.867 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:04:04.784969+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.869+0000 7f53be504640 1 -- 172.21.15.92:0/2603898611 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f53a406c8f0 msgr2=0x7f53a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.869+0000 7f53be504640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f53a406c8f0 0x7f53a406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f53b81507c0 tx=0x7f53a8008040 comp rx=0 tx=0).stop 2024-04-22T07:04:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.869+0000 7f53be504640 1 -- 172.21.15.92:0/2603898611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53b806d680 msgr2=0x7f53b80b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.869+0000 7f53be504640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53b806d680 0x7f53b80b2410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f53ac02b950 tx=0x7f53ac0046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.869+0000 7f53be504640 1 -- 172.21.15.92:0/2603898611 shutdown_connections 2024-04-22T07:04:25.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.869+0000 7f53be504640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f53a406c8f0 0x7f53a406edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.869+0000 7f53be504640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53b81583e0 0x7f53b8151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.869+0000 7f53be504640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53b8070e20 0x7f53b80b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.869+0000 7f53be504640 1 --2- 172.21.15.92:0/2603898611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53b806d680 0x7f53b80b2410 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.869+0000 7f53be504640 1 -- 172.21.15.92:0/2603898611 >> 172.21.15.92:0/2603898611 conn(0x7f53b806db80 msgr2=0x7f53b8158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:25.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.869+0000 7f53be504640 1 -- 172.21.15.92:0/2603898611 shutdown_connections 2024-04-22T07:04:25.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.869+0000 7f53be504640 1 -- 172.21.15.92:0/2603898611 wait complete. 2024-04-22T07:04:25.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:03:57.947601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:01.058111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:02.085602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:25.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:25.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.981+0000 7ffa1f63f640 1 Processor -- start 2024-04-22T07:04:25.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.981+0000 7ffa1f63f640 1 -- start start 2024-04-22T07:04:25.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.982+0000 7ffa1f63f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa180cd540 0x7ffa1815b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.982+0000 7ffa1f63f640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ffa180cd920 0x7ffa181547a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.982+0000 7ffa1f63f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffa180680b0 0x7ffa18154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.982+0000 7ffa1f63f640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7ffa180cac20 con 0x7ffa180680b0 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.982+0000 7ffa1f63f640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1815bb60 con 0x7ffa180cd920 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.982+0000 7ffa1f63f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffa180c9dc0 con 0x7ffa180cd540 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.982+0000 7ffa1dbdc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffa180680b0 0x7ffa18154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.982+0000 7ffa1dbdc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffa180680b0 0x7ffa18154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47318/0 (socket says 172.21.15.92:47318) 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.982+0000 7ffa1dbdc640 1 -- 172.21.15.92:0/1316724480 learned_addr learned my addr 172.21.15.92:0/1316724480 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.982+0000 7ffa1cbda640 1 --2- 172.21.15.92:0/1316724480 >> v2:172.21.15.145:3300/0 conn(0x7ffa180cd920 0x7ffa181547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.983+0000 7ffa1d3db640 1 --2- 172.21.15.92:0/1316724480 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa180cd540 0x7ffa1815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.983+0000 7ffa1dbdc640 1 -- 172.21.15.92:0/1316724480 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa180cd540 msgr2=0x7ffa1815b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.983+0000 7ffa1dbdc640 1 --2- 172.21.15.92:0/1316724480 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa180cd540 0x7ffa1815b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.983+0000 7ffa1dbdc640 1 -- 172.21.15.92:0/1316724480 >> v2:172.21.15.145:3300/0 conn(0x7ffa180cd920 msgr2=0x7ffa181547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.983+0000 7ffa1dbdc640 1 --2- 172.21.15.92:0/1316724480 >> v2:172.21.15.145:3300/0 conn(0x7ffa180cd920 0x7ffa181547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.983+0000 7ffa1dbdc640 1 -- 172.21.15.92:0/1316724480 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa180ca0a0 con 0x7ffa180680b0 2024-04-22T07:04:25.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.983+0000 7ffa1cbda640 1 --2- 172.21.15.92:0/1316724480 >> v2:172.21.15.145:3300/0 conn(0x7ffa180cd920 0x7ffa181547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:25.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.983+0000 7ffa1dbdc640 1 --2- 172.21.15.92:0/1316724480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180680b0 0x7ffa18154ce0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffa180caf00 tx=0x7ffa0c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4464e635e6f938b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:25.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.984+0000 7ffa0bfff640 1 -- 172.21.15.92:0/1316724480 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c02e070 con 0x7ffa180680b0 2024-04-22T07:04:25.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.984+0000 7ffa0bfff640 1 -- 172.21.15.92:0/1316724480 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0c037070 con 0x7ffa180680b0 2024-04-22T07:04:25.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.984+0000 7ffa0bfff640 1 -- 172.21.15.92:0/1316724480 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c031a00 con 0x7ffa180680b0 2024-04-22T07:04:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.984+0000 7ffa1f63f640 1 -- 172.21.15.92:0/1316724480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180680b0 msgr2=0x7ffa18154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.984+0000 7ffa1f63f640 1 --2- 172.21.15.92:0/1316724480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180680b0 0x7ffa18154ce0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffa180caf00 tx=0x7ffa0c02b880 comp rx=0 tx=0).stop 2024-04-22T07:04:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.984+0000 7ffa1f63f640 1 -- 172.21.15.92:0/1316724480 shutdown_connections 2024-04-22T07:04:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.984+0000 7ffa1f63f640 1 --2- 172.21.15.92:0/1316724480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180680b0 0x7ffa18154ce0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.984+0000 7ffa1f63f640 1 --2- 172.21.15.92:0/1316724480 >> v2:172.21.15.145:3300/0 conn(0x7ffa180cd920 0x7ffa181547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.984+0000 7ffa1f63f640 1 --2- 172.21.15.92:0/1316724480 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa180cd540 0x7ffa1815b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.984+0000 7ffa1f63f640 1 -- 172.21.15.92:0/1316724480 >> 172.21.15.92:0/1316724480 conn(0x7ffa18072ee0 msgr2=0x7ffa1806e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.984+0000 7ffa1f63f640 1 -- 172.21.15.92:0/1316724480 shutdown_connections 2024-04-22T07:04:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.984+0000 7ffa1f63f640 1 -- 172.21.15.92:0/1316724480 wait complete. 2024-04-22T07:04:25.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.985+0000 7ffa1f63f640 1 Processor -- start 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.986+0000 7ffa1f63f640 1 -- start start 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.986+0000 7ffa1f63f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa180680b0 0x7ffa180c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.986+0000 7ffa1f63f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa180cd540 0x7ffa1814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.986+0000 7ffa1f63f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180cd920 0x7ffa1814d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.986+0000 7ffa1f63f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa180ca940 con 0x7ffa180cd920 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.986+0000 7ffa1f63f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffa180ca660 con 0x7ffa180680b0 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.986+0000 7ffa1f63f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1806ba50 con 0x7ffa180cd540 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.986+0000 7ffa1dbdc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180cd920 0x7ffa1814d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.986+0000 7ffa1dbdc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180cd920 0x7ffa1814d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47326/0 (socket says 172.21.15.92:47326) 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.986+0000 7ffa1dbdc640 1 -- 172.21.15.92:0/422801454 learned_addr learned my addr 172.21.15.92:0/422801454 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.986+0000 7ffa1cbda640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa180cd540 0x7ffa1814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.986+0000 7ffa1d3db640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa180680b0 0x7ffa180c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.987+0000 7ffa1dbdc640 1 -- 172.21.15.92:0/422801454 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa180cd540 msgr2=0x7ffa1814cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.987+0000 7ffa1dbdc640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa180cd540 0x7ffa1814cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.987+0000 7ffa1dbdc640 1 -- 172.21.15.92:0/422801454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa180680b0 msgr2=0x7ffa180c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.987+0000 7ffa1dbdc640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa180680b0 0x7ffa180c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.987+0000 7ffa1dbdc640 1 -- 172.21.15.92:0/422801454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa18150e70 con 0x7ffa180cd920 2024-04-22T07:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.987+0000 7ffa1d3db640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa180680b0 0x7ffa180c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.987+0000 7ffa1dbdc640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180cd920 0x7ffa1814d3e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0090b0 tx=0x7ffa0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.987+0000 7ffa09ffb640 1 -- 172.21.15.92:0/422801454 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c032670 con 0x7ffa180cd920 2024-04-22T07:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.987+0000 7ffa09ffb640 1 -- 172.21.15.92:0/422801454 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0c032810 con 0x7ffa180cd920 2024-04-22T07:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.987+0000 7ffa1f63f640 1 -- 172.21.15.92:0/422801454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa180c41c0 con 0x7ffa180cd920 2024-04-22T07:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.987+0000 7ffa09ffb640 1 -- 172.21.15.92:0/422801454 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c032c00 con 0x7ffa180cd920 2024-04-22T07:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.987+0000 7ffa1f63f640 1 -- 172.21.15.92:0/422801454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa181513f0 con 0x7ffa180cd920 2024-04-22T07:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.988+0000 7ffa1f63f640 1 -- 172.21.15.92:0/422801454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa18071350 con 0x7ffa180cd920 2024-04-22T07:04:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.989+0000 7ffa09ffb640 1 -- 172.21.15.92:0/422801454 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa0c04c060 con 0x7ffa180cd920 2024-04-22T07:04:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.989+0000 7ffa09ffb640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff9fc06c8f0 0x7ff9fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.989+0000 7ffa09ffb640 1 -- 172.21.15.92:0/422801454 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffa0c0ad4f0 con 0x7ffa180cd920 2024-04-22T07:04:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.989+0000 7ffa1d3db640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff9fc06c8f0 0x7ff9fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:25.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.990+0000 7ffa1d3db640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff9fc06c8f0 0x7ff9fc06edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffa100074c0 tx=0x7ffa1000d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:25.993+0000 7ffa09ffb640 1 -- 172.21.15.92:0/422801454 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa18071350 con 0x7ffa180cd920 2024-04-22T07:04:26.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.176+0000 7ffa1f63f640 1 -- 172.21.15.92:0/422801454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffa18067b90 con 0x7ffa180cd920 2024-04-22T07:04:26.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.177+0000 7ffa09ffb640 1 -- 172.21.15.92:0/422801454 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7ffa18067b90 con 0x7ffa180cd920 2024-04-22T07:04:26.176 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-04-22T07:04:26.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.179+0000 7ffa1f63f640 1 -- 172.21.15.92:0/422801454 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff9fc06c8f0 msgr2=0x7ff9fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.179+0000 7ffa1f63f640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff9fc06c8f0 0x7ff9fc06edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffa100074c0 tx=0x7ffa1000d040 comp rx=0 tx=0).stop 2024-04-22T07:04:26.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.179+0000 7ffa1f63f640 1 -- 172.21.15.92:0/422801454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180cd920 msgr2=0x7ffa1814d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.179+0000 7ffa1f63f640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180cd920 0x7ffa1814d3e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0090b0 tx=0x7ffa0c0046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.179+0000 7ffa1f63f640 1 -- 172.21.15.92:0/422801454 shutdown_connections 2024-04-22T07:04:26.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.179+0000 7ffa1f63f640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff9fc06c8f0 0x7ff9fc06edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.179+0000 7ffa1f63f640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa180cd920 0x7ffa1814d3e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.179+0000 7ffa1f63f640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa180cd540 0x7ffa1814cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.179+0000 7ffa1f63f640 1 --2- 172.21.15.92:0/422801454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa180680b0 0x7ffa180c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.179+0000 7ffa1f63f640 1 -- 172.21.15.92:0/422801454 >> 172.21.15.92:0/422801454 conn(0x7ffa18072ee0 msgr2=0x7ffa1806be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:26.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.179+0000 7ffa1f63f640 1 -- 172.21.15.92:0/422801454 shutdown_connections 2024-04-22T07:04:26.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.179+0000 7ffa1f63f640 1 -- 172.21.15.92:0/422801454 wait complete. 2024-04-22T07:04:26.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:04:26.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.290+0000 7f503c606640 1 Processor -- start 2024-04-22T07:04:26.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.290+0000 7f503c606640 1 -- start start 2024-04-22T07:04:26.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f503c606640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f50340cd920 0x7f50341547a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f503c606640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50340680b0 0x7f5034154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f503c606640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50340cac20 con 0x7f50340cd540 2024-04-22T07:04:26.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f503c606640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f50340c9dc0 con 0x7f50340cd920 2024-04-22T07:04:26.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f503c606640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f50340ca0a0 con 0x7f50340680b0 2024-04-22T07:04:26.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f5039ba1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f50340cd920 0x7f50341547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f5039ba1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f50340cd920 0x7f50341547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47342/0 (socket says 172.21.15.92:47342) 2024-04-22T07:04:26.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f5039ba1640 1 -- 172.21.15.92:0/2495353871 learned_addr learned my addr 172.21.15.92:0/2495353871 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:26.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f503aba3640 1 --2- 172.21.15.92:0/2495353871 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50340680b0 0x7f5034154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f5039ba1640 1 -- 172.21.15.92:0/2495353871 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50340680b0 msgr2=0x7f5034154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:26.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f5039ba1640 1 --2- 172.21.15.92:0/2495353871 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50340680b0 0x7f5034154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f5039ba1640 1 -- 172.21.15.92:0/2495353871 >> v1:172.21.15.145:6789/0 conn(0x7f50340cd540 legacy=0x7f503415b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:26.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.291+0000 7f5039ba1640 1 -- 172.21.15.92:0/2495353871 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50340ca380 con 0x7f50340cd920 2024-04-22T07:04:26.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.292+0000 7f5039ba1640 1 --2- 172.21.15.92:0/2495353871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50340cd920 0x7f50341547a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5030009870 tx=0x7f503002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167c9b11c35f61ec server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:26.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.292+0000 7f50393a0640 1 -- 172.21.15.92:0/2495353871 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030035020 con 0x7f50340cd920 2024-04-22T07:04:26.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.292+0000 7f50393a0640 1 -- 172.21.15.92:0/2495353871 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5030031640 con 0x7f50340cd920 2024-04-22T07:04:26.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.292+0000 7f50393a0640 1 -- 172.21.15.92:0/2495353871 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503002d040 con 0x7f50340cd920 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.293+0000 7f503c606640 1 -- 172.21.15.92:0/2495353871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50340cd920 msgr2=0x7f50341547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.293+0000 7f503c606640 1 --2- 172.21.15.92:0/2495353871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50340cd920 0x7f50341547a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5030009870 tx=0x7f503002b8f0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.293+0000 7f503c606640 1 -- 172.21.15.92:0/2495353871 shutdown_connections 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.293+0000 7f503c606640 1 --2- 172.21.15.92:0/2495353871 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50340680b0 0x7f5034154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.293+0000 7f503c606640 1 --2- 172.21.15.92:0/2495353871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50340cd920 0x7f50341547a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.293+0000 7f503c606640 1 -- 172.21.15.92:0/2495353871 >> 172.21.15.92:0/2495353871 conn(0x7f5034072ee0 msgr2=0x7f503406e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.293+0000 7f503c606640 1 -- 172.21.15.92:0/2495353871 shutdown_connections 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.293+0000 7f503c606640 1 -- 172.21.15.92:0/2495353871 wait complete. 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.294+0000 7f503c606640 1 Processor -- start 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.294+0000 7f503c606640 1 -- start start 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503c606640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50340680b0 0x7f5034159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503c606640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50340cd540 0x7f5034155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503c606640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50340cd920 0x7f5034156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503c606640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f503406ba50 con 0x7f50340680b0 2024-04-22T07:04:26.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503c606640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f50340ca940 con 0x7f50340cd920 2024-04-22T07:04:26.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503c606640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f503406bbd0 con 0x7f50340cd540 2024-04-22T07:04:26.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503a3a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50340680b0 0x7f5034159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503a3a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50340680b0 0x7f5034159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47354/0 (socket says 172.21.15.92:47354) 2024-04-22T07:04:26.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503a3a2640 1 -- 172.21.15.92:0/282448129 learned_addr learned my addr 172.21.15.92:0/282448129 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:26.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f5039ba1640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50340cd540 0x7f5034155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503aba3640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50340cd920 0x7f5034156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503a3a2640 1 -- 172.21.15.92:0/282448129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50340cd540 msgr2=0x7f5034155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503a3a2640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50340cd540 0x7f5034155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503a3a2640 1 -- 172.21.15.92:0/282448129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50340cd920 msgr2=0x7f5034156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503a3a2640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50340cd920 0x7f5034156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503a3a2640 1 -- 172.21.15.92:0/282448129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5034153570 con 0x7f50340680b0 2024-04-22T07:04:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f503a3a2640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50340680b0 0x7f5034159ee0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f502400c430 tx=0x7f502400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f502affd640 1 -- 172.21.15.92:0/282448129 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50240052d0 con 0x7f50340680b0 2024-04-22T07:04:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f502affd640 1 -- 172.21.15.92:0/282448129 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5024005490 con 0x7f50340680b0 2024-04-22T07:04:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.295+0000 7f502affd640 1 -- 172.21.15.92:0/282448129 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024007260 con 0x7f50340680b0 2024-04-22T07:04:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.296+0000 7f503c606640 1 -- 172.21.15.92:0/282448129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50341f3b20 con 0x7f50340680b0 2024-04-22T07:04:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.296+0000 7f503c606640 1 -- 172.21.15.92:0/282448129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50341f40e0 con 0x7f50340680b0 2024-04-22T07:04:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.296+0000 7f503c606640 1 -- 172.21.15.92:0/282448129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5034068b00 con 0x7f50340680b0 2024-04-22T07:04:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.297+0000 7f502affd640 1 -- 172.21.15.92:0/282448129 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5024007400 con 0x7f50340680b0 2024-04-22T07:04:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.298+0000 7f502affd640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f501806c920 0x7f501806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.298+0000 7f5039ba1640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f501806c920 0x7f501806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.298+0000 7f502affd640 1 -- 172.21.15.92:0/282448129 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f502408b0b0 con 0x7f50340680b0 2024-04-22T07:04:26.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.298+0000 7f5039ba1640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f501806c920 0x7f501806ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5030036000 tx=0x7f503003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:26.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.302+0000 7f502affd640 1 -- 172.21.15.92:0/282448129 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5034153570 con 0x7f50340680b0 2024-04-22T07:04:26.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.480+0000 7f503c606640 1 -- 172.21.15.92:0/282448129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f503406d4c0 con 0x7f50340680b0 2024-04-22T07:04:26.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.481+0000 7f502affd640 1 -- 172.21.15.92:0/282448129 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f50240594e0 con 0x7f50340680b0 2024-04-22T07:04:26.480 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-04-22T07:04:26.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.483+0000 7f503c606640 1 -- 172.21.15.92:0/282448129 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f501806c920 msgr2=0x7f501806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.483+0000 7f503c606640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f501806c920 0x7f501806ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5030036000 tx=0x7f503003c040 comp rx=0 tx=0).stop 2024-04-22T07:04:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.483+0000 7f503c606640 1 -- 172.21.15.92:0/282448129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50340680b0 msgr2=0x7f5034159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.483+0000 7f503c606640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50340680b0 0x7f5034159ee0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f502400c430 tx=0x7f502400c8f0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.483+0000 7f503c606640 1 -- 172.21.15.92:0/282448129 shutdown_connections 2024-04-22T07:04:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.483+0000 7f503c606640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f501806c920 0x7f501806ede0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.483+0000 7f503c606640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50340cd920 0x7f5034156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.483+0000 7f503c606640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50340cd540 0x7f5034155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.483+0000 7f503c606640 1 --2- 172.21.15.92:0/282448129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50340680b0 0x7f5034159ee0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.483+0000 7f503c606640 1 -- 172.21.15.92:0/282448129 >> 172.21.15.92:0/282448129 conn(0x7f5034072ee0 msgr2=0x7f50340cb140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.483+0000 7f503c606640 1 -- 172.21.15.92:0/282448129 shutdown_connections 2024-04-22T07:04:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.483+0000 7f503c606640 1 -- 172.21.15.92:0/282448129 wait complete. 2024-04-22T07:04:26.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:04:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.593+0000 7f2b80d96640 1 Processor -- start 2024-04-22T07:04:26.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.593+0000 7f2b80d96640 1 -- start start 2024-04-22T07:04:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.594+0000 7f2b80d96640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2b7c15bee0 0x7f2b7c0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.594+0000 7f2b80d96640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2b7c0ce030 0x7f2b7c15c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.594+0000 7f2b80d96640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b7c15ca10 con 0x7f2b7c0c8f00 2024-04-22T07:04:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.594+0000 7f2b80d96640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f2b7c15ccf0 con 0x7f2b7c15bee0 2024-04-22T07:04:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.594+0000 7f2b80d96640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f2b7c0c54a0 con 0x7f2b7c0ce030 2024-04-22T07:04:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.594+0000 7f2b7a59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2b7c15bee0 0x7f2b7c0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.595+0000 7f2b7a59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2b7c15bee0 0x7f2b7c0cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47366/0 (socket says 172.21.15.92:47366) 2024-04-22T07:04:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.595+0000 7f2b7a59c640 1 -- 172.21.15.92:0/421584210 learned_addr learned my addr 172.21.15.92:0/421584210 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.595+0000 7f2b79d9b640 1 --2- 172.21.15.92:0/421584210 >> v2:172.21.15.145:3300/0 conn(0x7f2b7c0ce030 0x7f2b7c15c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.595+0000 7f2b7a59c640 1 -- 172.21.15.92:0/421584210 >> v2:172.21.15.145:3300/0 conn(0x7f2b7c0ce030 msgr2=0x7f2b7c15c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.595+0000 7f2b7a59c640 1 --2- 172.21.15.92:0/421584210 >> v2:172.21.15.145:3300/0 conn(0x7f2b7c0ce030 0x7f2b7c15c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.595+0000 7f2b7a59c640 1 -- 172.21.15.92:0/421584210 >> v1:172.21.15.145:6789/0 conn(0x7f2b7c0c8f00 legacy=0x7f2b7c0ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.595+0000 7f2b7a59c640 1 -- 172.21.15.92:0/421584210 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c0c36c0 con 0x7f2b7c15bee0 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.596+0000 7f2b7a59c640 1 --2- 172.21.15.92:0/421584210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b7c15bee0 0x7f2b7c0cd730 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0cf480 tx=0x7f2b7002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147d44d3066e34a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.596+0000 7f2b7959a640 1 -- 172.21.15.92:0/421584210 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70035020 con 0x7f2b7c15bee0 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.596+0000 7f2b7959a640 1 -- 172.21.15.92:0/421584210 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b70031870 con 0x7f2b7c15bee0 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.596+0000 7f2b7959a640 1 -- 172.21.15.92:0/421584210 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70031c40 con 0x7f2b7c15bee0 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.596+0000 7f2b80d96640 1 -- 172.21.15.92:0/421584210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b7c15bee0 msgr2=0x7f2b7c0cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.596+0000 7f2b80d96640 1 --2- 172.21.15.92:0/421584210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b7c15bee0 0x7f2b7c0cd730 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0cf480 tx=0x7f2b7002b910 comp rx=0 tx=0).stop 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.596+0000 7f2b80d96640 1 -- 172.21.15.92:0/421584210 shutdown_connections 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.596+0000 7f2b80d96640 1 --2- 172.21.15.92:0/421584210 >> v2:172.21.15.145:3300/0 conn(0x7f2b7c0ce030 0x7f2b7c15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.596+0000 7f2b80d96640 1 --2- 172.21.15.92:0/421584210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b7c15bee0 0x7f2b7c0cd730 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.596+0000 7f2b80d96640 1 -- 172.21.15.92:0/421584210 >> 172.21.15.92:0/421584210 conn(0x7f2b7c06df70 msgr2=0x7f2b7c06e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:26.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.596+0000 7f2b80d96640 1 -- 172.21.15.92:0/421584210 shutdown_connections 2024-04-22T07:04:26.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.596+0000 7f2b80d96640 1 -- 172.21.15.92:0/421584210 wait complete. 2024-04-22T07:04:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.598+0000 7f2b80d96640 1 Processor -- start 2024-04-22T07:04:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.598+0000 7f2b80d96640 1 -- start start 2024-04-22T07:04:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.598+0000 7f2b80d96640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b7c0c8f00 0x7f2b7c14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.598+0000 7f2b80d96640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b7c0ce030 0x7f2b7c14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.598+0000 7f2b80d96640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b7c15bee0 0x7f2b7c1ee980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.598+0000 7f2b80d96640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c0cd420 con 0x7f2b7c0ce030 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.598+0000 7f2b80d96640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c0c51a0 con 0x7f2b7c0c8f00 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.598+0000 7f2b80d96640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7c152f60 con 0x7f2b7c15bee0 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.598+0000 7f2b79d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b7c0ce030 0x7f2b7c14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.598+0000 7f2b79d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b7c0ce030 0x7f2b7c14f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47374/0 (socket says 172.21.15.92:47374) 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.598+0000 7f2b79d9b640 1 -- 172.21.15.92:0/2686317452 learned_addr learned my addr 172.21.15.92:0/2686317452 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b7ad9d640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b7c15bee0 0x7f2b7c1ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b7a59c640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b7c0c8f00 0x7f2b7c14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b79d9b640 1 -- 172.21.15.92:0/2686317452 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b7c15bee0 msgr2=0x7f2b7c1ee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b79d9b640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b7c15bee0 0x7f2b7c1ee980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b79d9b640 1 -- 172.21.15.92:0/2686317452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b7c0c8f00 msgr2=0x7f2b7c14e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b79d9b640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b7c0c8f00 0x7f2b7c14e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b79d9b640 1 -- 172.21.15.92:0/2686317452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c14d750 con 0x7f2b7c0ce030 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b79d9b640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b7c0ce030 0x7f2b7c14f4f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00c820 tx=0x7f2b6c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b6affd640 1 -- 172.21.15.92:0/2686317452 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c015070 con 0x7f2b7c0ce030 2024-04-22T07:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b6affd640 1 -- 172.21.15.92:0/2686317452 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c004240 con 0x7f2b7c0ce030 2024-04-22T07:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b6affd640 1 -- 172.21.15.92:0/2686317452 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c00ce10 con 0x7f2b7c0ce030 2024-04-22T07:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b80d96640 1 -- 172.21.15.92:0/2686317452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c070b20 con 0x7f2b7c0ce030 2024-04-22T07:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.599+0000 7f2b80d96640 1 -- 172.21.15.92:0/2686317452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c14d9f0 con 0x7f2b7c0ce030 2024-04-22T07:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.600+0000 7f2b80d96640 1 -- 172.21.15.92:0/2686317452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7c068b00 con 0x7f2b7c0ce030 2024-04-22T07:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.600+0000 7f2b6affd640 1 -- 172.21.15.92:0/2686317452 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b6c007830 con 0x7f2b7c0ce030 2024-04-22T07:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.601+0000 7f2b6affd640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f2b5c06c8f0 0x7f2b5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.601+0000 7f2b7a59c640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f2b5c06c8f0 0x7f2b5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.601+0000 7f2b6affd640 1 -- 172.21.15.92:0/2686317452 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2b6c08ba30 con 0x7f2b7c0ce030 2024-04-22T07:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.601+0000 7f2b7a59c640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f2b5c06c8f0 0x7f2b5c06edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2b70036000 tx=0x7f2b7003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:26.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.605+0000 7f2b6affd640 1 -- 172.21.15.92:0/2686317452 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b6c059e60 con 0x7f2b7c0ce030 2024-04-22T07:04:26.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.781+0000 7f2b80d96640 1 -- 172.21.15.92:0/2686317452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b7c0cac10 con 0x7f2b7c0ce030 2024-04-22T07:04:26.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.781+0000 7f2b6affd640 1 -- 172.21.15.92:0/2686317452 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f2b6c059760 con 0x7f2b7c0ce030 2024-04-22T07:04:26.781 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-04-22T07:04:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.783+0000 7f2b80d96640 1 -- 172.21.15.92:0/2686317452 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f2b5c06c8f0 msgr2=0x7f2b5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.783+0000 7f2b80d96640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f2b5c06c8f0 0x7f2b5c06edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2b70036000 tx=0x7f2b7003c040 comp rx=0 tx=0).stop 2024-04-22T07:04:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.783+0000 7f2b80d96640 1 -- 172.21.15.92:0/2686317452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b7c0ce030 msgr2=0x7f2b7c14f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.783+0000 7f2b80d96640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b7c0ce030 0x7f2b7c14f4f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00c820 tx=0x7f2b6c00cce0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.783+0000 7f2b80d96640 1 -- 172.21.15.92:0/2686317452 shutdown_connections 2024-04-22T07:04:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.783+0000 7f2b80d96640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f2b5c06c8f0 0x7f2b5c06edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.783+0000 7f2b80d96640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b7c15bee0 0x7f2b7c1ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.783+0000 7f2b80d96640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b7c0ce030 0x7f2b7c14f4f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.783+0000 7f2b80d96640 1 --2- 172.21.15.92:0/2686317452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b7c0c8f00 0x7f2b7c14e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.783+0000 7f2b80d96640 1 -- 172.21.15.92:0/2686317452 >> 172.21.15.92:0/2686317452 conn(0x7f2b7c06df70 msgr2=0x7f2b7c0c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.783+0000 7f2b80d96640 1 -- 172.21.15.92:0/2686317452 shutdown_connections 2024-04-22T07:04:26.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.784+0000 7f2b80d96640 1 -- 172.21.15.92:0/2686317452 wait complete. 2024-04-22T07:04:26.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.898+0000 7fb6ebf2b640 1 Processor -- start 2024-04-22T07:04:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.898+0000 7fb6ebf2b640 1 -- start start 2024-04-22T07:04:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.899+0000 7fb6ebf2b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb6e4156190 0x7fb6e4070490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.899+0000 7fb6ebf2b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6e40720e0 con 0x7fb6e40c8df0 2024-04-22T07:04:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.899+0000 7fb6ebf2b640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6e4072350 con 0x7fb6e40c43c0 2024-04-22T07:04:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.899+0000 7fb6ebf2b640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb6e40c3410 con 0x7fb6e4156190 2024-04-22T07:04:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.899+0000 7fb6e94c6640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb6e40c8df0 0x7fb6e40c3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44984/0 (socket says 172.21.15.92:44984) 2024-04-22T07:04:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.899+0000 7fb6e94c6640 1 -- 172.21.15.92:0/3765858689 learned_addr learned my addr 172.21.15.92:0/3765858689 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.899+0000 7fb6e9cc7640 1 --2- 172.21.15.92:0/3765858689 >> v2:172.21.15.145:3300/0 conn(0x7fb6e4156190 0x7fb6e4070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.899+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 715379136 0 0) 0x7fb6e40720e0 con 0x7fb6e40c8df0 2024-04-22T07:04:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.899+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6d0003520 con 0x7fb6e40c8df0 2024-04-22T07:04:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1856070254 0 0) 0x7fb6d0003520 con 0x7fb6e40c8df0 2024-04-22T07:04:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 >> v2:172.21.15.145:3300/0 conn(0x7fb6e4156190 msgr2=0x7fb6e4070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 --2- 172.21.15.92:0/3765858689 >> v2:172.21.15.145:3300/0 conn(0x7fb6e4156190 0x7fb6e4070490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 >> v1:172.21.15.145:6789/0 conn(0x7fb6e40c43c0 legacy=0x7fb6e4163330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e40c3660 con 0x7fb6e40c8df0 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7fb6d4001690 con 0x7fb6e40c8df0 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 >> v1:172.21.15.92:6789/0 conn(0x7fb6e40c8df0 legacy=0x7fb6e40c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 --2- 172.21.15.92:0/3765858689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6d00039b0 0x7fb6d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 --2- 172.21.15.92:0/3765858689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6d0004800 0x7fb6d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 --2- 172.21.15.92:0/3765858689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb6d00051c0 0x7fb6d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e40c3410 con 0x7fb6d00039b0 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d000c060 con 0x7fb6d0004800 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d000c260 con 0x7fb6d00051c0 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e9cc7640 1 --2- 172.21.15.92:0/3765858689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6d00039b0 0x7fb6d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6ea4c8640 1 --2- 172.21.15.92:0/3765858689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb6d00051c0 0x7fb6d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e94c6640 1 --2- 172.21.15.92:0/3765858689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6d0004800 0x7fb6d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e9cc7640 1 -- 172.21.15.92:0/3765858689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb6d00051c0 msgr2=0x7fb6d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e9cc7640 1 --2- 172.21.15.92:0/3765858689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb6d00051c0 0x7fb6d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e9cc7640 1 -- 172.21.15.92:0/3765858689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6d0004800 msgr2=0x7fb6d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e9cc7640 1 --2- 172.21.15.92:0/3765858689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6d0004800 0x7fb6d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6e9cc7640 1 -- 172.21.15.92:0/3765858689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d0003520 con 0x7fb6d00039b0 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.900+0000 7fb6ea4c8640 1 -- 172.21.15.92:0/3765858689 reap_dead start 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.901+0000 7fb6e9cc7640 1 --2- 172.21.15.92:0/3765858689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6d00039b0 0x7fb6d0003dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009ac0 tx=0x7fb6d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751c1827ba7d33ef server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.901+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0039070 con 0x7fb6d00039b0 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.901+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e002c6d0 con 0x7fb6d00039b0 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.901+0000 7fb6e8cc5640 1 -- 172.21.15.92:0/3765858689 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0009690 con 0x7fb6d00039b0 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.901+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/3765858689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6d00039b0 msgr2=0x7fb6d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.901+0000 7fb6ebf2b640 1 --2- 172.21.15.92:0/3765858689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6d00039b0 0x7fb6d0003dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009ac0 tx=0x7fb6d0005830 comp rx=0 tx=0).stop 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.901+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/3765858689 shutdown_connections 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.901+0000 7fb6ebf2b640 1 --2- 172.21.15.92:0/3765858689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6d00039b0 0x7fb6d0003dd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.901+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/3765858689 >> 172.21.15.92:0/3765858689 conn(0x7fb6e406e260 msgr2=0x7fb6e40715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:26.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.902+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/3765858689 shutdown_connections 2024-04-22T07:04:26.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.902+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/3765858689 wait complete. 2024-04-22T07:04:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.903+0000 7fb6ebf2b640 1 Processor -- start 2024-04-22T07:04:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.903+0000 7fb6ebf2b640 1 -- start start 2024-04-22T07:04:26.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.903+0000 7fb6ebf2b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6d00039b0 0x7fb6e40ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.903+0000 7fb6ebf2b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb6e40cce20 0x7fb6e40cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.903+0000 7fb6ebf2b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e41e35e0 0x7fb6e41e3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.903+0000 7fb6ebf2b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e406f9b0 con 0x7fb6e41e35e0 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.903+0000 7fb6ebf2b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4070000 con 0x7fb6d00039b0 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.903+0000 7fb6ebf2b640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e406fc90 con 0x7fb6e40cce20 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.903+0000 7fb6ea4c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e41e35e0 0x7fb6e41e3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.903+0000 7fb6ea4c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e41e35e0 0x7fb6e41e3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47398/0 (socket says 172.21.15.92:47398) 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.903+0000 7fb6ea4c8640 1 -- 172.21.15.92:0/1618286078 learned_addr learned my addr 172.21.15.92:0/1618286078 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6e94c6640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb6e40cce20 0x7fb6e40cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6e9cc7640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6d00039b0 0x7fb6e40ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6ea4c8640 1 -- 172.21.15.92:0/1618286078 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb6e40cce20 msgr2=0x7fb6e40cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6ea4c8640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb6e40cce20 0x7fb6e40cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6ea4c8640 1 -- 172.21.15.92:0/1618286078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6d00039b0 msgr2=0x7fb6e40ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6ea4c8640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6d00039b0 0x7fb6e40ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6ea4c8640 1 -- 172.21.15.92:0/1618286078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e41e3fa0 con 0x7fb6e41e35e0 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6ea4c8640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e41e35e0 0x7fb6e41e3a60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40b1af0 tx=0x7fb6dc00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6da7fc640 1 -- 172.21.15.92:0/1618286078 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc003970 con 0x7fb6e41e35e0 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6da7fc640 1 -- 172.21.15.92:0/1618286078 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc018070 con 0x7fb6e41e35e0 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/1618286078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e40cd350 con 0x7fb6e41e35e0 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6da7fc640 1 -- 172.21.15.92:0/1618286078 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc003d70 con 0x7fb6e41e35e0 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.904+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/1618286078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e40cd910 con 0x7fb6e41e35e0 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.905+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/1618286078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e4068b00 con 0x7fb6e41e35e0 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.905+0000 7fb6da7fc640 1 -- 172.21.15.92:0/1618286078 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6dc0050d0 con 0x7fb6e41e35e0 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.906+0000 7fb6da7fc640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb6c806c8f0 0x7fb6c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.906+0000 7fb6e9cc7640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb6c806c8f0 0x7fb6c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.906+0000 7fb6da7fc640 1 -- 172.21.15.92:0/1618286078 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb6dc08b5c0 con 0x7fb6e41e35e0 2024-04-22T07:04:26.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.906+0000 7fb6e9cc7640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb6c806c8f0 0x7fb6c806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0036000 tx=0x7fb6e0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:26.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:26.910+0000 7fb6da7fc640 1 -- 172.21.15.92:0/1618286078 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6e41e3fa0 con 0x7fb6e41e35e0 2024-04-22T07:04:27.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.151+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/1618286078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6e41e2f20 con 0x7fb6e41e35e0 2024-04-22T07:04:27.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.151+0000 7fb6da7fc640 1 -- 172.21.15.92:0/1618286078 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6909 (secure 0 0 0) 0x7fb6dc0593a0 con 0x7fb6e41e35e0 2024-04-22T07:04:27.150 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:27.151 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1059009649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1059009649},{"type":"v1","addr":"172.21.15.145:6835","nonce":1059009649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:03.129095+0000","modified":"2024-04-22T07:04:25.017461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4275,"mds_1":4273,"mds_2":4269,"mds_3":4232,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.145:6836/1058270486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1058270486},{"type":"v1","addr":"172.21.15.145:6836","nonce":1058270486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4232":{"gid":4232,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.145:6837/3258302920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3258302920},{"type":"v1","addr":"172.21.15.145:6837","nonce":3258302920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/101798872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":101798872},{"type":"v1","addr":"172.21.15.92:6837","nonce":101798872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6839/3493022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3493022212},{"type":"v1","addr":"172.21.15.92:6839","nonce":3493022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275,4273,4269,4232]},"id":1}]} 2024-04-22T07:04:27.151 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 16 2024-04-22T07:04:27.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.153+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/1618286078 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb6c806c8f0 msgr2=0x7fb6c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.153+0000 7fb6ebf2b640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb6c806c8f0 0x7fb6c806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0036000 tx=0x7fb6e0030000 comp rx=0 tx=0).stop 2024-04-22T07:04:27.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.153+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/1618286078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e41e35e0 msgr2=0x7fb6e41e3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.153+0000 7fb6ebf2b640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e41e35e0 0x7fb6e41e3a60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40b1af0 tx=0x7fb6dc00eb90 comp rx=0 tx=0).stop 2024-04-22T07:04:27.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.153+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/1618286078 shutdown_connections 2024-04-22T07:04:27.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.153+0000 7fb6ebf2b640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb6c806c8f0 0x7fb6c806edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.153+0000 7fb6ebf2b640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e41e35e0 0x7fb6e41e3a60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.153+0000 7fb6ebf2b640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb6e40cce20 0x7fb6e40cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.153+0000 7fb6ebf2b640 1 --2- 172.21.15.92:0/1618286078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6d00039b0 0x7fb6e40ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.153+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/1618286078 >> 172.21.15.92:0/1618286078 conn(0x7fb6e406e260 msgr2=0x7fb6e4154350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:27.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.153+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/1618286078 shutdown_connections 2024-04-22T07:04:27.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.153+0000 7fb6ebf2b640 1 -- 172.21.15.92:0/1618286078 wait complete. 2024-04-22T07:04:27.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:27.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.262+0000 7ff59d0e0640 1 Processor -- start 2024-04-22T07:04:27.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.262+0000 7ff59d0e0640 1 -- start start 2024-04-22T07:04:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.263+0000 7ff59d0e0640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff598070db0 0x7ff59806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.263+0000 7ff59d0e0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5980c8ce0 0x7ff59806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.263+0000 7ff59d0e0640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5980cec70 con 0x7ff598155240 2024-04-22T07:04:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.263+0000 7ff59d0e0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7ff5980ce3d0 con 0x7ff5980c8ce0 2024-04-22T07:04:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.263+0000 7ff59d0e0640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff5980ce6b0 con 0x7ff598070db0 2024-04-22T07:04:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.263+0000 7ff59759e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5980c8ce0 0x7ff59806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.263+0000 7ff59759e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5980c8ce0 0x7ff59806beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47408/0 (socket says 172.21.15.92:47408) 2024-04-22T07:04:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.263+0000 7ff59759e640 1 -- 172.21.15.92:0/4089662656 learned_addr learned my addr 172.21.15.92:0/4089662656 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.263+0000 7ff58ffff640 1 --2- 172.21.15.92:0/4089662656 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff598070db0 0x7ff59806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.264+0000 7ff59759e640 1 -- 172.21.15.92:0/4089662656 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff598070db0 msgr2=0x7ff59806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.264+0000 7ff59759e640 1 --2- 172.21.15.92:0/4089662656 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff598070db0 0x7ff59806b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.264+0000 7ff59759e640 1 -- 172.21.15.92:0/4089662656 >> v1:172.21.15.145:6789/0 conn(0x7ff598155240 legacy=0x7ff59815b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.264+0000 7ff59759e640 1 -- 172.21.15.92:0/4089662656 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5980ce990 con 0x7ff5980c8ce0 2024-04-22T07:04:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.265+0000 7ff59759e640 1 --2- 172.21.15.92:0/4089662656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5980c8ce0 0x7ff59806beb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff590009e50 tx=0x7ff59002bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dec4a17d0640e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.265+0000 7ff59659c640 1 -- 172.21.15.92:0/4089662656 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59002cb90 con 0x7ff5980c8ce0 2024-04-22T07:04:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.265+0000 7ff59659c640 1 -- 172.21.15.92:0/4089662656 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59002cd30 con 0x7ff5980c8ce0 2024-04-22T07:04:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.265+0000 7ff59659c640 1 -- 172.21.15.92:0/4089662656 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590034610 con 0x7ff5980c8ce0 2024-04-22T07:04:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.265+0000 7ff59d0e0640 1 -- 172.21.15.92:0/4089662656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5980c8ce0 msgr2=0x7ff59806beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.265+0000 7ff59d0e0640 1 --2- 172.21.15.92:0/4089662656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5980c8ce0 0x7ff59806beb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff590009e50 tx=0x7ff59002bff0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.265+0000 7ff59d0e0640 1 -- 172.21.15.92:0/4089662656 shutdown_connections 2024-04-22T07:04:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.265+0000 7ff59d0e0640 1 --2- 172.21.15.92:0/4089662656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5980c8ce0 0x7ff59806beb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.265+0000 7ff59d0e0640 1 --2- 172.21.15.92:0/4089662656 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff598070db0 0x7ff59806b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.265+0000 7ff59d0e0640 1 -- 172.21.15.92:0/4089662656 >> 172.21.15.92:0/4089662656 conn(0x7ff598067ce0 msgr2=0x7ff5980714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.265+0000 7ff59d0e0640 1 -- 172.21.15.92:0/4089662656 shutdown_connections 2024-04-22T07:04:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.265+0000 7ff59d0e0640 1 -- 172.21.15.92:0/4089662656 wait complete. 2024-04-22T07:04:27.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.267+0000 7ff59d0e0640 1 Processor -- start 2024-04-22T07:04:27.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.267+0000 7ff59d0e0640 1 -- start start 2024-04-22T07:04:27.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.267+0000 7ff59d0e0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5980c8ce0 0x7ff5980c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.267+0000 7ff59d0e0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff598155240 0x7ff5980c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.267+0000 7ff59d0e0640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5980c5be0 0x7ff5980c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.267+0000 7ff59d0e0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff5980708f0 con 0x7ff5980c8ce0 2024-04-22T07:04:27.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.267+0000 7ff59d0e0640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff59815b150 con 0x7ff598155240 2024-04-22T07:04:27.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.267+0000 7ff59d0e0640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff5980ca1d0 con 0x7ff5980c5be0 2024-04-22T07:04:27.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.267+0000 7ff596d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5980c8ce0 0x7ff5980c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff596d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5980c8ce0 0x7ff5980c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47410/0 (socket says 172.21.15.92:47410) 2024-04-22T07:04:27.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff596d9d640 1 -- 172.21.15.92:0/3841013894 learned_addr learned my addr 172.21.15.92:0/3841013894 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:27.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff59759e640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5980c5be0 0x7ff5980c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff58ffff640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff598155240 0x7ff5980c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff596d9d640 1 -- 172.21.15.92:0/3841013894 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5980c5be0 msgr2=0x7ff5980c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff596d9d640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5980c5be0 0x7ff5980c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff596d9d640 1 -- 172.21.15.92:0/3841013894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff598155240 msgr2=0x7ff5980c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff596d9d640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff598155240 0x7ff5980c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff596d9d640 1 -- 172.21.15.92:0/3841013894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5981558c0 con 0x7ff5980c8ce0 2024-04-22T07:04:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff59759e640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5980c5be0 0x7ff5980c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff596d9d640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5980c8ce0 0x7ff5980c7ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff58800c3f0 tx=0x7ff58800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff58f7fe640 1 -- 172.21.15.92:0/3841013894 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880052d0 con 0x7ff5980c8ce0 2024-04-22T07:04:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff58f7fe640 1 -- 172.21.15.92:0/3841013894 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff588005490 con 0x7ff5980c8ce0 2024-04-22T07:04:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff59d0e0640 1 -- 172.21.15.92:0/3841013894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff598155b60 con 0x7ff5980c8ce0 2024-04-22T07:04:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff58f7fe640 1 -- 172.21.15.92:0/3841013894 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588007260 con 0x7ff5980c8ce0 2024-04-22T07:04:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.268+0000 7ff59d0e0640 1 -- 172.21.15.92:0/3841013894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5981f1c30 con 0x7ff5980c8ce0 2024-04-22T07:04:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.269+0000 7ff59d0e0640 1 -- 172.21.15.92:0/3841013894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff598153f50 con 0x7ff5980c8ce0 2024-04-22T07:04:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.270+0000 7ff58f7fe640 1 -- 172.21.15.92:0/3841013894 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff588007400 con 0x7ff5980c8ce0 2024-04-22T07:04:27.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.270+0000 7ff58f7fe640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff57806c920 0x7ff57806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.270+0000 7ff58f7fe640 1 -- 172.21.15.92:0/3841013894 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff58808b080 con 0x7ff5980c8ce0 2024-04-22T07:04:27.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.270+0000 7ff58ffff640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff57806c920 0x7ff57806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.271+0000 7ff58ffff640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff57806c920 0x7ff57806ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff5980b2690 tx=0x7ff580008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:27.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.278+0000 7ff58f7fe640 1 -- 172.21.15.92:0/3841013894 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff598153f50 con 0x7ff5980c8ce0 2024-04-22T07:04:27.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.460+0000 7ff59d0e0640 1 -- 172.21.15.92:0/3841013894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5981f1e70 con 0x7ff5980c8ce0 2024-04-22T07:04:27.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.462+0000 7ff58f7fe640 1 -- 172.21.15.92:0/3841013894 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7ff588059420 con 0x7ff5980c8ce0 2024-04-22T07:04:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:27.460 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:04:04.784969+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:27.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.464+0000 7ff59d0e0640 1 -- 172.21.15.92:0/3841013894 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff57806c920 msgr2=0x7ff57806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.464+0000 7ff59d0e0640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff57806c920 0x7ff57806ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff5980b2690 tx=0x7ff580008040 comp rx=0 tx=0).stop 2024-04-22T07:04:27.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.464+0000 7ff59d0e0640 1 -- 172.21.15.92:0/3841013894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5980c8ce0 msgr2=0x7ff5980c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.464+0000 7ff59d0e0640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5980c8ce0 0x7ff5980c7ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff58800c3f0 tx=0x7ff58800c8b0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.464+0000 7ff59d0e0640 1 -- 172.21.15.92:0/3841013894 shutdown_connections 2024-04-22T07:04:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.464+0000 7ff59d0e0640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff57806c920 0x7ff57806ede0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.464+0000 7ff59d0e0640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5980c5be0 0x7ff5980c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.464+0000 7ff59d0e0640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff598155240 0x7ff5980c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.464+0000 7ff59d0e0640 1 --2- 172.21.15.92:0/3841013894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5980c8ce0 0x7ff5980c7ea0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.464+0000 7ff59d0e0640 1 -- 172.21.15.92:0/3841013894 >> 172.21.15.92:0/3841013894 conn(0x7ff598067ce0 msgr2=0x7ff598153840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.464+0000 7ff59d0e0640 1 -- 172.21.15.92:0/3841013894 shutdown_connections 2024-04-22T07:04:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.464+0000 7ff59d0e0640 1 -- 172.21.15.92:0/3841013894 wait complete. 2024-04-22T07:04:27.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:03:57.947601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:01.058111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:02.085602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:27.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:27.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.575+0000 7f6b4b19d640 1 Processor -- start 2024-04-22T07:04:27.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.575+0000 7f6b4b19d640 1 -- start start 2024-04-22T07:04:27.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b4b19d640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6b440ce380 0x7f6b440ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b4b19d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6b44070db0 0x7f6b440c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b4b19d640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b440c54a0 con 0x7f6b440c3950 2024-04-22T07:04:27.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b4b19d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f6b440ca4d0 con 0x7f6b44070db0 2024-04-22T07:04:27.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b4b19d640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6b440c8e90 con 0x7f6b440ce380 2024-04-22T07:04:27.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b43fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6b44070db0 0x7f6b440c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b43fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6b44070db0 0x7f6b440c3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39108/0 (socket says 172.21.15.92:39108) 2024-04-22T07:04:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b43fff640 1 -- 172.21.15.92:0/1719073873 learned_addr learned my addr 172.21.15.92:0/1719073873 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b48f39640 1 --2- 172.21.15.92:0/1719073873 >> v2:172.21.15.145:3300/0 conn(0x7f6b440ce380 0x7f6b440ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b43fff640 1 -- 172.21.15.92:0/1719073873 >> v2:172.21.15.145:3300/0 conn(0x7f6b440ce380 msgr2=0x7f6b440ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b43fff640 1 --2- 172.21.15.92:0/1719073873 >> v2:172.21.15.145:3300/0 conn(0x7f6b440ce380 0x7f6b440ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b43fff640 1 -- 172.21.15.92:0/1719073873 >> v1:172.21.15.145:6789/0 conn(0x7f6b440c3950 legacy=0x7f6b4415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.576+0000 7f6b43fff640 1 -- 172.21.15.92:0/1719073873 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b440c9170 con 0x7f6b44070db0 2024-04-22T07:04:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.577+0000 7f6b43fff640 1 --2- 172.21.15.92:0/1719073873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b44070db0 0x7f6b440c3410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6b440cefc0 tx=0x7f6b3002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d659d397eff53e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.577+0000 7f6b437fe640 1 -- 172.21.15.92:0/1719073873 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b300317c0 con 0x7f6b44070db0 2024-04-22T07:04:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.577+0000 7f6b437fe640 1 -- 172.21.15.92:0/1719073873 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b30037070 con 0x7f6b44070db0 2024-04-22T07:04:27.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.577+0000 7f6b437fe640 1 -- 172.21.15.92:0/1719073873 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30031b80 con 0x7f6b44070db0 2024-04-22T07:04:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.578+0000 7f6b4b19d640 1 -- 172.21.15.92:0/1719073873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b44070db0 msgr2=0x7f6b440c3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.578+0000 7f6b4b19d640 1 --2- 172.21.15.92:0/1719073873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b44070db0 0x7f6b440c3410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6b440cefc0 tx=0x7f6b3002b860 comp rx=0 tx=0).stop 2024-04-22T07:04:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.578+0000 7f6b4b19d640 1 -- 172.21.15.92:0/1719073873 shutdown_connections 2024-04-22T07:04:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.578+0000 7f6b4b19d640 1 --2- 172.21.15.92:0/1719073873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b44070db0 0x7f6b440c3410 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.578+0000 7f6b4b19d640 1 --2- 172.21.15.92:0/1719073873 >> v2:172.21.15.145:3300/0 conn(0x7f6b440ce380 0x7f6b440ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.578+0000 7f6b4b19d640 1 -- 172.21.15.92:0/1719073873 >> 172.21.15.92:0/1719073873 conn(0x7f6b4406db80 msgr2=0x7f6b4406df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.578+0000 7f6b4b19d640 1 -- 172.21.15.92:0/1719073873 shutdown_connections 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.578+0000 7f6b4b19d640 1 -- 172.21.15.92:0/1719073873 wait complete. 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.579+0000 7f6b4b19d640 1 Processor -- start 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.579+0000 7f6b4b19d640 1 -- start start 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4b19d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b44070db0 0x7f6b4415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4b19d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b440c3950 0x7f6b441ef360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4b19d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b440ce380 0x7f6b440b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4b19d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6b44070260 con 0x7f6b440ce380 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4b19d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6b44159740 con 0x7f6b44070db0 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4b19d640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6b440c5190 con 0x7f6b440c3950 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4973a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b440ce380 0x7f6b440b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4973a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b440ce380 0x7f6b440b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39112/0 (socket says 172.21.15.92:39112) 2024-04-22T07:04:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4973a640 1 -- 172.21.15.92:0/2901569956 learned_addr learned my addr 172.21.15.92:0/2901569956 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b43fff640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b440c3950 0x7f6b441ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b48f39640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b44070db0 0x7f6b4415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4973a640 1 -- 172.21.15.92:0/2901569956 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b440c3950 msgr2=0x7f6b441ef360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4973a640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b440c3950 0x7f6b441ef360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4973a640 1 -- 172.21.15.92:0/2901569956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b44070db0 msgr2=0x7f6b4415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4973a640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b44070db0 0x7f6b4415ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4973a640 1 -- 172.21.15.92:0/2901569956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4415aa70 con 0x7f6b440ce380 2024-04-22T07:04:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b4973a640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b440ce380 0x7f6b440b2880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00b970 tx=0x7f6b3c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.580+0000 7f6b417fa640 1 -- 172.21.15.92:0/2901569956 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c019070 con 0x7f6b440ce380 2024-04-22T07:04:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.581+0000 7f6b417fa640 1 -- 172.21.15.92:0/2901569956 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b3c00cb40 con 0x7f6b440ce380 2024-04-22T07:04:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.581+0000 7f6b417fa640 1 -- 172.21.15.92:0/2901569956 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c014440 con 0x7f6b440ce380 2024-04-22T07:04:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.581+0000 7f6b4b19d640 1 -- 172.21.15.92:0/2901569956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4415ad10 con 0x7f6b440ce380 2024-04-22T07:04:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.581+0000 7f6b4b19d640 1 -- 172.21.15.92:0/2901569956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b440b16e0 con 0x7f6b440ce380 2024-04-22T07:04:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.582+0000 7f6b4b19d640 1 -- 172.21.15.92:0/2901569956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b44068b00 con 0x7f6b440ce380 2024-04-22T07:04:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.582+0000 7f6b417fa640 1 -- 172.21.15.92:0/2901569956 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b3c01c070 con 0x7f6b440ce380 2024-04-22T07:04:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.583+0000 7f6b417fa640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f6b2806c8f0 0x7f6b2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.583+0000 7f6b48f39640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f6b2806c8f0 0x7f6b2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.583+0000 7f6b417fa640 1 -- 172.21.15.92:0/2901569956 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6b3c08b820 con 0x7f6b440ce380 2024-04-22T07:04:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.583+0000 7f6b48f39640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f6b2806c8f0 0x7f6b2806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6b38006fd0 tx=0x7f6b38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:27.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.587+0000 7f6b417fa640 1 -- 172.21.15.92:0/2901569956 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b3c059bc0 con 0x7f6b440ce380 2024-04-22T07:04:27.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.767+0000 7f6b4b19d640 1 -- 172.21.15.92:0/2901569956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6b440c3e50 con 0x7f6b440ce380 2024-04-22T07:04:27.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.768+0000 7f6b417fa640 1 -- 172.21.15.92:0/2901569956 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f6b3c0594c0 con 0x7f6b440ce380 2024-04-22T07:04:27.766 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-04-22T07:04:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.769+0000 7f6b4b19d640 1 -- 172.21.15.92:0/2901569956 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f6b2806c8f0 msgr2=0x7f6b2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.769+0000 7f6b4b19d640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f6b2806c8f0 0x7f6b2806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6b38006fd0 tx=0x7f6b38008040 comp rx=0 tx=0).stop 2024-04-22T07:04:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.769+0000 7f6b4b19d640 1 -- 172.21.15.92:0/2901569956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b440ce380 msgr2=0x7f6b440b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.770+0000 7f6b4b19d640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b440ce380 0x7f6b440b2880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00b970 tx=0x7f6b3c00be30 comp rx=0 tx=0).stop 2024-04-22T07:04:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.770+0000 7f6b4b19d640 1 -- 172.21.15.92:0/2901569956 shutdown_connections 2024-04-22T07:04:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.770+0000 7f6b4b19d640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f6b2806c8f0 0x7f6b2806edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.770+0000 7f6b4b19d640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b440ce380 0x7f6b440b2880 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.770+0000 7f6b4b19d640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b440c3950 0x7f6b441ef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.770+0000 7f6b4b19d640 1 --2- 172.21.15.92:0/2901569956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b44070db0 0x7f6b4415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.770+0000 7f6b4b19d640 1 -- 172.21.15.92:0/2901569956 >> 172.21.15.92:0/2901569956 conn(0x7f6b4406db80 msgr2=0x7f6b44152120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.770+0000 7f6b4b19d640 1 -- 172.21.15.92:0/2901569956 shutdown_connections 2024-04-22T07:04:27.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.770+0000 7f6b4b19d640 1 -- 172.21.15.92:0/2901569956 wait complete. 2024-04-22T07:04:27.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:04:27.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.882+0000 7f894b1f4640 1 Processor -- start 2024-04-22T07:04:27.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.883+0000 7f894b1f4640 1 -- start start 2024-04-22T07:04:27.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.883+0000 7f894b1f4640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8944154f30 0x7f894415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.883+0000 7f894b1f4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894415ea00 0x7f89440cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.883+0000 7f894b1f4640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8944155310 con 0x7f894415ee40 2024-04-22T07:04:27.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.883+0000 7f894b1f4640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f894415b230 con 0x7f8944154f30 2024-04-22T07:04:27.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.883+0000 7f894b1f4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f894415b510 con 0x7f894415ea00 2024-04-22T07:04:27.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.883+0000 7f8949791640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f894415ee40 0x7f89440ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53032/0 (socket says 172.21.15.92:53032) 2024-04-22T07:04:27.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.883+0000 7f8949791640 1 -- 172.21.15.92:0/2562345898 learned_addr learned my addr 172.21.15.92:0/2562345898 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:27.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893bfff640 1 --2- 172.21.15.92:0/2562345898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894415ea00 0x7f89440cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f8948f90640 1 --2- 172.21.15.92:0/2562345898 >> v2:172.21.15.145:3300/0 conn(0x7f8944154f30 0x7f894415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1663017694 0 0) 0x7f8944155310 con 0x7f894415ee40 2024-04-22T07:04:27.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f892c003520 con 0x7f894415ee40 2024-04-22T07:04:27.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 799035028 0 0) 0x7f892c003520 con 0x7f894415ee40 2024-04-22T07:04:27.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894415ea00 msgr2=0x7f89440cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:27.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 --2- 172.21.15.92:0/2562345898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894415ea00 0x7f89440cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 >> v2:172.21.15.145:3300/0 conn(0x7f8944154f30 msgr2=0x7f894415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:27.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 --2- 172.21.15.92:0/2562345898 >> v2:172.21.15.145:3300/0 conn(0x7f8944154f30 0x7f894415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894415b7f0 con 0x7f894415ee40 2024-04-22T07:04:27.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f893c0019f0 con 0x7f894415ee40 2024-04-22T07:04:27.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 >> v1:172.21.15.92:6789/0 conn(0x7f894415ee40 legacy=0x7f89440ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 --2- 172.21.15.92:0/2562345898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f892c0039b0 0x7f892c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 --2- 172.21.15.92:0/2562345898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894415ea00 0x7f892c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 --2- 172.21.15.92:0/2562345898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f892c004d40 0x7f892c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f894415b230 con 0x7f892c004d40 2024-04-22T07:04:27.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f894415b510 con 0x7f892c0039b0 2024-04-22T07:04:27.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f892c00b9c0 con 0x7f894415ea00 2024-04-22T07:04:27.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f8949791640 1 --2- 172.21.15.92:0/2562345898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f892c004d40 0x7f892c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f8948f90640 1 --2- 172.21.15.92:0/2562345898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f892c0039b0 0x7f892c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f893bfff640 1 --2- 172.21.15.92:0/2562345898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894415ea00 0x7f892c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f8949791640 1 -- 172.21.15.92:0/2562345898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894415ea00 msgr2=0x7f892c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:27.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.884+0000 7f8949791640 1 --2- 172.21.15.92:0/2562345898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894415ea00 0x7f892c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.885+0000 7f8949791640 1 -- 172.21.15.92:0/2562345898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f892c0039b0 msgr2=0x7f892c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:27.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.885+0000 7f8949791640 1 --2- 172.21.15.92:0/2562345898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f892c0039b0 0x7f892c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.885+0000 7f8949791640 1 -- 172.21.15.92:0/2562345898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c003520 con 0x7f892c004d40 2024-04-22T07:04:27.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.885+0000 7f8949791640 1 --2- 172.21.15.92:0/2562345898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f892c004d40 0x7f892c0051c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f893c0013d0 tx=0x7f892c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53b51ce5e07fae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:27.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.885+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c00a4e0 con 0x7f892c004d40 2024-04-22T07:04:27.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.885+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f893c034650 con 0x7f892c004d40 2024-04-22T07:04:27.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.885+0000 7f893b7fe640 1 -- 172.21.15.92:0/2562345898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c034a40 con 0x7f892c004d40 2024-04-22T07:04:27.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.886+0000 7f894b1f4640 1 -- 172.21.15.92:0/2562345898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f892c004d40 msgr2=0x7f892c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.886+0000 7f894b1f4640 1 --2- 172.21.15.92:0/2562345898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f892c004d40 0x7f892c0051c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f893c0013d0 tx=0x7f892c005830 comp rx=0 tx=0).stop 2024-04-22T07:04:27.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.886+0000 7f8949791640 1 -- 172.21.15.92:0/2562345898 reap_dead start 2024-04-22T07:04:27.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.886+0000 7f894b1f4640 1 -- 172.21.15.92:0/2562345898 shutdown_connections 2024-04-22T07:04:27.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.886+0000 7f894b1f4640 1 -- 172.21.15.92:0/2562345898 >> 172.21.15.92:0/2562345898 conn(0x7f8944073ca0 msgr2=0x7f894406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:27.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.886+0000 7f894b1f4640 1 -- 172.21.15.92:0/2562345898 shutdown_connections 2024-04-22T07:04:27.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.886+0000 7f894b1f4640 1 -- 172.21.15.92:0/2562345898 wait complete. 2024-04-22T07:04:27.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.887+0000 7f894b1f4640 1 Processor -- start 2024-04-22T07:04:27.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.887+0000 7f894b1f4640 1 -- start start 2024-04-22T07:04:27.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f894b1f4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f892c004d40 0x7f894415d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f894b1f4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f894415dd70 0x7f894414fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f894b1f4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894414f1d0 0x7f8944150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f894b1f4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f894415e4e0 con 0x7f892c004d40 2024-04-22T07:04:27.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f894b1f4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f894415e200 con 0x7f894415dd70 2024-04-22T07:04:27.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f894b1f4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f89440c58a0 con 0x7f894414f1d0 2024-04-22T07:04:27.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f8948f90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f892c004d40 0x7f894415d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f8948f90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f892c004d40 0x7f894415d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39136/0 (socket says 172.21.15.92:39136) 2024-04-22T07:04:27.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f8948f90640 1 -- 172.21.15.92:0/657837513 learned_addr learned my addr 172.21.15.92:0/657837513 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:27.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f893bfff640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f894415dd70 0x7f894414fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f8949791640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894414f1d0 0x7f8944150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f8948f90640 1 -- 172.21.15.92:0/657837513 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894414f1d0 msgr2=0x7f8944150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f8948f90640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894414f1d0 0x7f8944150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f8948f90640 1 -- 172.21.15.92:0/657837513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f894415dd70 msgr2=0x7f894414fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f8948f90640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f894415dd70 0x7f894414fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f8948f90640 1 -- 172.21.15.92:0/657837513 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89441f0900 con 0x7f892c004d40 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f8949791640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894414f1d0 0x7f8944150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f893bfff640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f894415dd70 0x7f894414fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.888+0000 7f8948f90640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f892c004d40 0x7f894415d2a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89400119c0 tx=0x7f8940011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.889+0000 7f89397fa640 1 -- 172.21.15.92:0/657837513 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894000fb50 con 0x7f892c004d40 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.889+0000 7f89397fa640 1 -- 172.21.15.92:0/657837513 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f894000fcf0 con 0x7f892c004d40 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.889+0000 7f89397fa640 1 -- 172.21.15.92:0/657837513 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894000fea0 con 0x7f892c004d40 2024-04-22T07:04:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.889+0000 7f894b1f4640 1 -- 172.21.15.92:0/657837513 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89441f0be0 con 0x7f892c004d40 2024-04-22T07:04:27.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.889+0000 7f894b1f4640 1 -- 172.21.15.92:0/657837513 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89441f11a0 con 0x7f892c004d40 2024-04-22T07:04:27.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.890+0000 7f894b1f4640 1 -- 172.21.15.92:0/657837513 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89440b3830 con 0x7f892c004d40 2024-04-22T07:04:27.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.890+0000 7f89397fa640 1 -- 172.21.15.92:0/657837513 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8940012040 con 0x7f892c004d40 2024-04-22T07:04:27.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.890+0000 7f89397fa640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f893006c8f0 0x7f893006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:27.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.891+0000 7f89397fa640 1 -- 172.21.15.92:0/657837513 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f894008c250 con 0x7f892c004d40 2024-04-22T07:04:27.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.891+0000 7f893bfff640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f893006c8f0 0x7f893006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:27.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.891+0000 7f893bfff640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f893006c8f0 0x7f893006edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f89440b62f0 tx=0x7f8934008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:27.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:27.895+0000 7f89397fa640 1 -- 172.21.15.92:0/657837513 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89440b3830 con 0x7f892c004d40 2024-04-22T07:04:27.912 INFO:tasks.ceph.mds.e.smithi092.stderr:2024-04-22T07:04:27.914+0000 7ff94f4d7640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-22T07:04:28.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.092+0000 7f894b1f4640 1 -- 172.21.15.92:0/657837513 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8944069930 con 0x7f892c004d40 2024-04-22T07:04:28.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.092+0000 7f89397fa640 1 -- 172.21.15.92:0/657837513 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f8940019030 con 0x7f892c004d40 2024-04-22T07:04:28.090 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-04-22T07:04:28.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.094+0000 7f894b1f4640 1 -- 172.21.15.92:0/657837513 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f893006c8f0 msgr2=0x7f893006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.094+0000 7f894b1f4640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f893006c8f0 0x7f893006edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f89440b62f0 tx=0x7f8934008040 comp rx=0 tx=0).stop 2024-04-22T07:04:28.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.094+0000 7f894b1f4640 1 -- 172.21.15.92:0/657837513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f892c004d40 msgr2=0x7f894415d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.094+0000 7f894b1f4640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f892c004d40 0x7f894415d2a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89400119c0 tx=0x7f8940011e80 comp rx=0 tx=0).stop 2024-04-22T07:04:28.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.094+0000 7f894b1f4640 1 -- 172.21.15.92:0/657837513 shutdown_connections 2024-04-22T07:04:28.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.094+0000 7f894b1f4640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f893006c8f0 0x7f893006edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.094+0000 7f894b1f4640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f894414f1d0 0x7f8944150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.094+0000 7f894b1f4640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f894415dd70 0x7f894414fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.094+0000 7f894b1f4640 1 --2- 172.21.15.92:0/657837513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f892c004d40 0x7f894415d2a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.094+0000 7f894b1f4640 1 -- 172.21.15.92:0/657837513 >> 172.21.15.92:0/657837513 conn(0x7f8944073ca0 msgr2=0x7f8944158f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:28.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.094+0000 7f894b1f4640 1 -- 172.21.15.92:0/657837513 shutdown_connections 2024-04-22T07:04:28.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.094+0000 7f894b1f4640 1 -- 172.21.15.92:0/657837513 wait complete. 2024-04-22T07:04:28.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:04:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.205+0000 7f414303f640 1 Processor -- start 2024-04-22T07:04:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.205+0000 7f414303f640 1 -- start start 2024-04-22T07:04:28.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f414303f640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f413c1551e0 0x7f413c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f414303f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c0c5870 0x7f413c15acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f414303f640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f413c15b3f0 con 0x7f413c15ebd0 2024-04-22T07:04:28.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f414303f640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f413c15bc90 con 0x7f413c1551e0 2024-04-22T07:04:28.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f414303f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f413c15b6d0 con 0x7f413c0c5870 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f4140ddb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f413c1551e0 0x7f413c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f413bfff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c0c5870 0x7f413c15acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f41415dc640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f413c15ebd0 0x7f413c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:48100/0 (socket says 172.21.15.92:48100) 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f41415dc640 1 -- 172.21.15.92:0/1190553374 learned_addr learned my addr 172.21.15.92:0/1190553374 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f413bfff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c0c5870 0x7f413c15acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:33938/0 (socket says 172.21.15.92:33938) 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f4140ddb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f413c1551e0 0x7f413c0cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:38970/0 (socket says 172.21.15.92:38970) 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f413b7fe640 1 -- 172.21.15.92:0/1190553374 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1101773355 0 0) 0x7f413c15b3f0 con 0x7f413c15ebd0 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.206+0000 7f413b7fe640 1 -- 172.21.15.92:0/1190553374 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f412c003520 con 0x7f413c15ebd0 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.207+0000 7f4140ddb640 1 -- 172.21.15.92:0/1190553374 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c0c5870 msgr2=0x7f413c15acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.207+0000 7f4140ddb640 1 --2- 172.21.15.92:0/1190553374 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c0c5870 0x7f413c15acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.207+0000 7f4140ddb640 1 -- 172.21.15.92:0/1190553374 >> v1:172.21.15.145:6789/0 conn(0x7f413c15ebd0 legacy=0x7f413c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.207+0000 7f4140ddb640 1 -- 172.21.15.92:0/1190553374 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c15b9b0 con 0x7f413c1551e0 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.207+0000 7f413bfff640 1 --2- 172.21.15.92:0/1190553374 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c0c5870 0x7f413c15acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.207+0000 7f4140ddb640 1 --2- 172.21.15.92:0/1190553374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c1551e0 0x7f413c0cc950 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f41300097a0 tx=0x7f413002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e03a289893997db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:28.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.207+0000 7f413b7fe640 1 -- 172.21.15.92:0/1190553374 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130035020 con 0x7f413c1551e0 2024-04-22T07:04:28.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.208+0000 7f413b7fe640 1 -- 172.21.15.92:0/1190553374 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4130031830 con 0x7f413c1551e0 2024-04-22T07:04:28.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.208+0000 7f413b7fe640 1 -- 172.21.15.92:0/1190553374 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130031be0 con 0x7f413c1551e0 2024-04-22T07:04:28.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.208+0000 7f414303f640 1 -- 172.21.15.92:0/1190553374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c1551e0 msgr2=0x7f413c0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.208+0000 7f414303f640 1 --2- 172.21.15.92:0/1190553374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c1551e0 0x7f413c0cc950 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f41300097a0 tx=0x7f413002b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.208+0000 7f414303f640 1 -- 172.21.15.92:0/1190553374 shutdown_connections 2024-04-22T07:04:28.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.208+0000 7f414303f640 1 --2- 172.21.15.92:0/1190553374 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c0c5870 0x7f413c15acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.208+0000 7f414303f640 1 --2- 172.21.15.92:0/1190553374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c1551e0 0x7f413c0cc950 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.208+0000 7f414303f640 1 -- 172.21.15.92:0/1190553374 >> 172.21.15.92:0/1190553374 conn(0x7f413c0d05b0 msgr2=0x7f413c0d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.208+0000 7f414303f640 1 -- 172.21.15.92:0/1190553374 shutdown_connections 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.208+0000 7f414303f640 1 -- 172.21.15.92:0/1190553374 wait complete. 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.209+0000 7f414303f640 1 Processor -- start 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.209+0000 7f414303f640 1 -- start start 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.210+0000 7f414303f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c0c5870 0x7f413c149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.210+0000 7f414303f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f413c1551e0 0x7f413c1451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.210+0000 7f414303f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c15ebd0 0x7f413c145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.210+0000 7f414303f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f413c0c5eb0 con 0x7f413c1551e0 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.210+0000 7f414303f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f413c0c5d30 con 0x7f413c0c5870 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.210+0000 7f414303f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f413c06c080 con 0x7f413c15ebd0 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.210+0000 7f413bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f413c1551e0 0x7f413c1451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.210+0000 7f413bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f413c1551e0 0x7f413c1451e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39144/0 (socket says 172.21.15.92:39144) 2024-04-22T07:04:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.210+0000 7f413bfff640 1 -- 172.21.15.92:0/2663831602 learned_addr learned my addr 172.21.15.92:0/2663831602 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.210+0000 7f41415dc640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c15ebd0 0x7f413c145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.210+0000 7f4140ddb640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c0c5870 0x7f413c149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.211+0000 7f413bfff640 1 -- 172.21.15.92:0/2663831602 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c15ebd0 msgr2=0x7f413c145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.211+0000 7f413bfff640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c15ebd0 0x7f413c145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.211+0000 7f413bfff640 1 -- 172.21.15.92:0/2663831602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c0c5870 msgr2=0x7f413c149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.211+0000 7f413bfff640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c0c5870 0x7f413c149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.211+0000 7f413bfff640 1 -- 172.21.15.92:0/2663831602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c15aa20 con 0x7f413c1551e0 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.211+0000 7f413bfff640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f413c1551e0 0x7f413c1451e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f412800e910 tx=0x7f412800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.211+0000 7f41397fa640 1 -- 172.21.15.92:0/2663831602 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128017070 con 0x7f413c1551e0 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.211+0000 7f41397fa640 1 -- 172.21.15.92:0/2663831602 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412800ca00 con 0x7f413c1551e0 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.211+0000 7f414303f640 1 -- 172.21.15.92:0/2663831602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413c15cc10 con 0x7f413c1551e0 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.211+0000 7f41397fa640 1 -- 172.21.15.92:0/2663831602 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412800cdb0 con 0x7f413c1551e0 2024-04-22T07:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.211+0000 7f414303f640 1 -- 172.21.15.92:0/2663831602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413c159c60 con 0x7f413c1551e0 2024-04-22T07:04:28.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.212+0000 7f414303f640 1 -- 172.21.15.92:0/2663831602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f413c068b00 con 0x7f413c1551e0 2024-04-22T07:04:28.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.212+0000 7f41397fa640 1 -- 172.21.15.92:0/2663831602 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f412801a070 con 0x7f413c1551e0 2024-04-22T07:04:28.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.213+0000 7f41397fa640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f412006c8f0 0x7f412006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.213+0000 7f41397fa640 1 -- 172.21.15.92:0/2663831602 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f412808ba60 con 0x7f413c1551e0 2024-04-22T07:04:28.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.213+0000 7f4140ddb640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f412006c8f0 0x7f412006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.213+0000 7f4140ddb640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f412006c8f0 0x7f412006edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4130036000 tx=0x7f413003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.217+0000 7f41397fa640 1 -- 172.21.15.92:0/2663831602 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4128059730 con 0x7f413c1551e0 2024-04-22T07:04:28.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.398+0000 7f414303f640 1 -- 172.21.15.92:0/2663831602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f413c0b3f60 con 0x7f413c1551e0 2024-04-22T07:04:28.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.398+0000 7f41397fa640 1 -- 172.21.15.92:0/2663831602 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f412805da30 con 0x7f413c1551e0 2024-04-22T07:04:28.396 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-04-22T07:04:28.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.400+0000 7f414303f640 1 -- 172.21.15.92:0/2663831602 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f412006c8f0 msgr2=0x7f412006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.400+0000 7f414303f640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f412006c8f0 0x7f412006edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4130036000 tx=0x7f413003c040 comp rx=0 tx=0).stop 2024-04-22T07:04:28.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.400+0000 7f414303f640 1 -- 172.21.15.92:0/2663831602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f413c1551e0 msgr2=0x7f413c1451e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.400+0000 7f414303f640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f413c1551e0 0x7f413c1451e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f412800e910 tx=0x7f412800edd0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.400+0000 7f414303f640 1 -- 172.21.15.92:0/2663831602 shutdown_connections 2024-04-22T07:04:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.400+0000 7f414303f640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f412006c8f0 0x7f412006edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.400+0000 7f414303f640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c15ebd0 0x7f413c145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.400+0000 7f414303f640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f413c1551e0 0x7f413c1451e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.400+0000 7f414303f640 1 --2- 172.21.15.92:0/2663831602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c0c5870 0x7f413c149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.400+0000 7f414303f640 1 -- 172.21.15.92:0/2663831602 >> 172.21.15.92:0/2663831602 conn(0x7f413c0d05b0 msgr2=0x7f413c0ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.400+0000 7f414303f640 1 -- 172.21.15.92:0/2663831602 shutdown_connections 2024-04-22T07:04:28.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.400+0000 7f414303f640 1 -- 172.21.15.92:0/2663831602 wait complete. 2024-04-22T07:04:28.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.509+0000 7fb3edb32640 1 Processor -- start 2024-04-22T07:04:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.509+0000 7fb3edb32640 1 -- start start 2024-04-22T07:04:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3edb32640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb3e80cd540 0x7fb3e806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3edb32640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3e80cd920 0x7fb3e8151770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3edb32640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e806d250 con 0x7fb3e80c8440 2024-04-22T07:04:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3edb32640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e80c91b0 con 0x7fb3e80cd920 2024-04-22T07:04:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3edb32640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e80c8910 con 0x7fb3e80cd540 2024-04-22T07:04:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3e7fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3e80cd920 0x7fb3e8151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3dffff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb3e80c8440 0x7fb3e80c4840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53044/0 (socket says 172.21.15.92:53044) 2024-04-22T07:04:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3dffff640 1 -- 172.21.15.92:0/2844313217 learned_addr learned my addr 172.21.15.92:0/2844313217 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3e77fe640 1 --2- 172.21.15.92:0/2844313217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb3e80cd540 0x7fb3e806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3e7fff640 1 -- 172.21.15.92:0/2844313217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb3e80cd540 msgr2=0x7fb3e806cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3e7fff640 1 --2- 172.21.15.92:0/2844313217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb3e80cd540 0x7fb3e806cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3e7fff640 1 -- 172.21.15.92:0/2844313217 >> v1:172.21.15.92:6789/0 conn(0x7fb3e80c8440 legacy=0x7fb3e80c4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3e7fff640 1 -- 172.21.15.92:0/2844313217 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e80c8bf0 con 0x7fb3e80cd920 2024-04-22T07:04:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3e6ffd640 1 -- 172.21.15.92:0/2844313217 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2879771067 0 0) 0x7fb3e806d250 con 0x7fb3e80c8440 2024-04-22T07:04:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.510+0000 7fb3e77fe640 1 --2- 172.21.15.92:0/2844313217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb3e80cd540 0x7fb3e806cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.511+0000 7fb3e7fff640 1 --2- 172.21.15.92:0/2844313217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e80cd920 0x7fb3e8151770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009d90 tx=0x7fb3e002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98c312c3fb37c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.512+0000 7fb3e6ffd640 1 -- 172.21.15.92:0/2844313217 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0039070 con 0x7fb3e80cd920 2024-04-22T07:04:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.512+0000 7fb3e6ffd640 1 -- 172.21.15.92:0/2844313217 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e002cce0 con 0x7fb3e80cd920 2024-04-22T07:04:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.512+0000 7fb3e6ffd640 1 -- 172.21.15.92:0/2844313217 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0034660 con 0x7fb3e80cd920 2024-04-22T07:04:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.512+0000 7fb3edb32640 1 -- 172.21.15.92:0/2844313217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e80cd920 msgr2=0x7fb3e8151770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.512+0000 7fb3edb32640 1 --2- 172.21.15.92:0/2844313217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e80cd920 0x7fb3e8151770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009d90 tx=0x7fb3e002bf30 comp rx=0 tx=0).stop 2024-04-22T07:04:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.512+0000 7fb3edb32640 1 -- 172.21.15.92:0/2844313217 shutdown_connections 2024-04-22T07:04:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.512+0000 7fb3edb32640 1 --2- 172.21.15.92:0/2844313217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e80cd920 0x7fb3e8151770 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.512+0000 7fb3edb32640 1 --2- 172.21.15.92:0/2844313217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb3e80cd540 0x7fb3e806cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.512+0000 7fb3edb32640 1 -- 172.21.15.92:0/2844313217 >> 172.21.15.92:0/2844313217 conn(0x7fb3e8072ee0 msgr2=0x7fb3e806e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.512+0000 7fb3edb32640 1 -- 172.21.15.92:0/2844313217 shutdown_connections 2024-04-22T07:04:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.512+0000 7fb3edb32640 1 -- 172.21.15.92:0/2844313217 wait complete. 2024-04-22T07:04:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.513+0000 7fb3edb32640 1 Processor -- start 2024-04-22T07:04:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.513+0000 7fb3edb32640 1 -- start start 2024-04-22T07:04:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3edb32640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb3e80c8440 0x7fb3e80cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3edb32640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3e80cd540 0x7fb3e814b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3edb32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e80cd920 0x7fb3e814b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3edb32640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e806c7f0 con 0x7fb3e80cd920 2024-04-22T07:04:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3edb32640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e806c510 con 0x7fb3e80cd540 2024-04-22T07:04:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3edb32640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e80c7e20 con 0x7fb3e80c8440 2024-04-22T07:04:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e80cd920 0x7fb3e814b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e80cd920 0x7fb3e814b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39154/0 (socket says 172.21.15.92:39154) 2024-04-22T07:04:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3e7fff640 1 -- 172.21.15.92:0/378820303 learned_addr learned my addr 172.21.15.92:0/378820303 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3e77fe640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb3e80c8440 0x7fb3e80cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3dffff640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3e80cd540 0x7fb3e814b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3e7fff640 1 -- 172.21.15.92:0/378820303 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb3e80c8440 msgr2=0x7fb3e80cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3e7fff640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb3e80c8440 0x7fb3e80cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3e7fff640 1 -- 172.21.15.92:0/378820303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3e80cd540 msgr2=0x7fb3e814b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3e7fff640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3e80cd540 0x7fb3e814b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.514+0000 7fb3e7fff640 1 -- 172.21.15.92:0/378820303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e814af10 con 0x7fb3e80cd920 2024-04-22T07:04:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.515+0000 7fb3e7fff640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e80cd920 0x7fb3e814b930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009d90 tx=0x7fb3e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.515+0000 7fb3e4ff9640 1 -- 172.21.15.92:0/378820303 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0039050 con 0x7fb3e80cd920 2024-04-22T07:04:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.515+0000 7fb3e4ff9640 1 -- 172.21.15.92:0/378820303 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e0045070 con 0x7fb3e80cd920 2024-04-22T07:04:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.515+0000 7fb3edb32640 1 -- 172.21.15.92:0/378820303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e814a030 con 0x7fb3e80cd920 2024-04-22T07:04:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.515+0000 7fb3e4ff9640 1 -- 172.21.15.92:0/378820303 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e003e850 con 0x7fb3e80cd920 2024-04-22T07:04:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.515+0000 7fb3edb32640 1 -- 172.21.15.92:0/378820303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e814a5f0 con 0x7fb3e80cd920 2024-04-22T07:04:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.516+0000 7fb3edb32640 1 -- 172.21.15.92:0/378820303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e814b0e0 con 0x7fb3e80cd920 2024-04-22T07:04:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.516+0000 7fb3e4ff9640 1 -- 172.21.15.92:0/378820303 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3e003e9f0 con 0x7fb3e80cd920 2024-04-22T07:04:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.517+0000 7fb3e4ff9640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb3c806c8f0 0x7fb3c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.517+0000 7fb3e4ff9640 1 -- 172.21.15.92:0/378820303 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb3e00ae930 con 0x7fb3e80cd920 2024-04-22T07:04:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.517+0000 7fb3e77fe640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb3c806c8f0 0x7fb3c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.517+0000 7fb3e77fe640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb3c806c8f0 0x7fb3c806edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800a990 tx=0x7fb3d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.521+0000 7fb3e4ff9640 1 -- 172.21.15.92:0/378820303 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3e007cf40 con 0x7fb3e80cd920 2024-04-22T07:04:28.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.739+0000 7fb3edb32640 1 -- 172.21.15.92:0/378820303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3e8068b00 con 0x7fb3e80cd920 2024-04-22T07:04:28.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.739+0000 7fb3e4ff9640 1 -- 172.21.15.92:0/378820303 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6909 (secure 0 0 0) 0x7fb3e007c840 con 0x7fb3e80cd920 2024-04-22T07:04:28.738 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:28.738 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1059009649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1059009649},{"type":"v1","addr":"172.21.15.145:6835","nonce":1059009649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:03.129095+0000","modified":"2024-04-22T07:04:25.017461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4275,"mds_1":4273,"mds_2":4269,"mds_3":4232,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.145:6836/1058270486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1058270486},{"type":"v1","addr":"172.21.15.145:6836","nonce":1058270486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4232":{"gid":4232,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.145:6837/3258302920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3258302920},{"type":"v1","addr":"172.21.15.145:6837","nonce":3258302920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/101798872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":101798872},{"type":"v1","addr":"172.21.15.92:6837","nonce":101798872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6839/3493022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3493022212},{"type":"v1","addr":"172.21.15.92:6839","nonce":3493022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275,4273,4269,4232]},"id":1}]} 2024-04-22T07:04:28.738 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 16 2024-04-22T07:04:28.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.741+0000 7fb3edb32640 1 -- 172.21.15.92:0/378820303 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb3c806c8f0 msgr2=0x7fb3c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.741+0000 7fb3edb32640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb3c806c8f0 0x7fb3c806edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800a990 tx=0x7fb3d8008040 comp rx=0 tx=0).stop 2024-04-22T07:04:28.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.741+0000 7fb3edb32640 1 -- 172.21.15.92:0/378820303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e80cd920 msgr2=0x7fb3e814b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.741+0000 7fb3edb32640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e80cd920 0x7fb3e814b930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009d90 tx=0x7fb3e0004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:28.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.741+0000 7fb3edb32640 1 -- 172.21.15.92:0/378820303 shutdown_connections 2024-04-22T07:04:28.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.741+0000 7fb3edb32640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb3c806c8f0 0x7fb3c806edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.741+0000 7fb3edb32640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e80cd920 0x7fb3e814b930 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.741+0000 7fb3edb32640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3e80cd540 0x7fb3e814b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.741+0000 7fb3edb32640 1 --2- 172.21.15.92:0/378820303 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb3e80c8440 0x7fb3e80cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.741+0000 7fb3edb32640 1 -- 172.21.15.92:0/378820303 >> 172.21.15.92:0/378820303 conn(0x7fb3e8072ee0 msgr2=0x7fb3e814f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:28.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.741+0000 7fb3edb32640 1 -- 172.21.15.92:0/378820303 shutdown_connections 2024-04-22T07:04:28.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.741+0000 7fb3edb32640 1 -- 172.21.15.92:0/378820303 wait complete. 2024-04-22T07:04:28.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:28.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.852+0000 7fb98e3d5640 1 Processor -- start 2024-04-22T07:04:28.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.852+0000 7fb98e3d5640 1 -- start start 2024-04-22T07:04:28.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb98e3d5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb988152dd0 0x7fb988070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb98e3d5640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9880c9970 0x7fb9880c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb98e3d5640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb988071270 con 0x7fb98806b970 2024-04-22T07:04:28.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb98e3d5640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fb988070840 con 0x7fb988152dd0 2024-04-22T07:04:28.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb98e3d5640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb9880700b0 con 0x7fb9880c9970 2024-04-22T07:04:28.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb987fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb988152dd0 0x7fb988070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb987fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb988152dd0 0x7fb988070be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39162/0 (socket says 172.21.15.92:39162) 2024-04-22T07:04:28.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb987fff640 1 -- 172.21.15.92:0/599939059 learned_addr learned my addr 172.21.15.92:0/599939059 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:28.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb987fff640 1 -- 172.21.15.92:0/599939059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9880c9970 msgr2=0x7fb9880c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:28.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb9877fe640 1 --2- 172.21.15.92:0/599939059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9880c9970 0x7fb9880c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb987fff640 1 --2- 172.21.15.92:0/599939059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9880c9970 0x7fb9880c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb987fff640 1 -- 172.21.15.92:0/599939059 >> v1:172.21.15.145:6789/0 conn(0x7fb98806b970 legacy=0x7fb9881589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb987fff640 1 -- 172.21.15.92:0/599939059 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9880c7e00 con 0x7fb988152dd0 2024-04-22T07:04:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.853+0000 7fb9877fe640 1 --2- 172.21.15.92:0/599939059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9880c9970 0x7fb9880c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.854+0000 7fb987fff640 1 --2- 172.21.15.92:0/599939059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb988152dd0 0x7fb988070be0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009870 tx=0x7fb97c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5cad04cb555834 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.854+0000 7fb986ffd640 1 -- 172.21.15.92:0/599939059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c035020 con 0x7fb988152dd0 2024-04-22T07:04:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.854+0000 7fb986ffd640 1 -- 172.21.15.92:0/599939059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c031640 con 0x7fb988152dd0 2024-04-22T07:04:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.854+0000 7fb986ffd640 1 -- 172.21.15.92:0/599939059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c02d040 con 0x7fb988152dd0 2024-04-22T07:04:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.854+0000 7fb98e3d5640 1 -- 172.21.15.92:0/599939059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb988152dd0 msgr2=0x7fb988070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.854+0000 7fb98e3d5640 1 --2- 172.21.15.92:0/599939059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb988152dd0 0x7fb988070be0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009870 tx=0x7fb97c02b8f0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.854+0000 7fb98e3d5640 1 -- 172.21.15.92:0/599939059 shutdown_connections 2024-04-22T07:04:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.854+0000 7fb98e3d5640 1 --2- 172.21.15.92:0/599939059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9880c9970 0x7fb9880c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.854+0000 7fb98e3d5640 1 --2- 172.21.15.92:0/599939059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb988152dd0 0x7fb988070be0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.854+0000 7fb98e3d5640 1 -- 172.21.15.92:0/599939059 >> 172.21.15.92:0/599939059 conn(0x7fb9880cd2d0 msgr2=0x7fb9880cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:28.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.855+0000 7fb98e3d5640 1 -- 172.21.15.92:0/599939059 shutdown_connections 2024-04-22T07:04:28.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.855+0000 7fb98e3d5640 1 -- 172.21.15.92:0/599939059 wait complete. 2024-04-22T07:04:28.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.856+0000 7fb98e3d5640 1 Processor -- start 2024-04-22T07:04:28.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.856+0000 7fb98e3d5640 1 -- start start 2024-04-22T07:04:28.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.856+0000 7fb98e3d5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb98806b970 0x7fb98814c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.856+0000 7fb98e3d5640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9880c9970 0x7fb9880c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.856+0000 7fb98e3d5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb988152dd0 0x7fb9881f0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.856+0000 7fb98e3d5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb9880c83c0 con 0x7fb98806b970 2024-04-22T07:04:28.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.856+0000 7fb98e3d5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb98806fd40 con 0x7fb988152dd0 2024-04-22T07:04:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.856+0000 7fb98e3d5640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb9881590e0 con 0x7fb9880c9970 2024-04-22T07:04:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb987fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb98806b970 0x7fb98814c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb987fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb98806b970 0x7fb98814c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39176/0 (socket says 172.21.15.92:39176) 2024-04-22T07:04:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb987fff640 1 -- 172.21.15.92:0/991857006 learned_addr learned my addr 172.21.15.92:0/991857006 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb987fff640 1 -- 172.21.15.92:0/991857006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9880c9970 msgr2=0x7fb9880c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb987fff640 1 --2- 172.21.15.92:0/991857006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9880c9970 0x7fb9880c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb987fff640 1 -- 172.21.15.92:0/991857006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb988152dd0 msgr2=0x7fb9881f0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb987fff640 1 --2- 172.21.15.92:0/991857006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb988152dd0 0x7fb9881f0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb987fff640 1 -- 172.21.15.92:0/991857006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98814c860 con 0x7fb98806b970 2024-04-22T07:04:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb987fff640 1 --2- 172.21.15.92:0/991857006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb98806b970 0x7fb98814c320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb97c02b930 tx=0x7fb97c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb984ff9640 1 -- 172.21.15.92:0/991857006 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c040070 con 0x7fb98806b970 2024-04-22T07:04:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb984ff9640 1 -- 172.21.15.92:0/991857006 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c039c10 con 0x7fb98806b970 2024-04-22T07:04:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb984ff9640 1 -- 172.21.15.92:0/991857006 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c035040 con 0x7fb98806b970 2024-04-22T07:04:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb98e3d5640 1 -- 172.21.15.92:0/991857006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9880c5eb0 con 0x7fb98806b970 2024-04-22T07:04:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.857+0000 7fb98e3d5640 1 -- 172.21.15.92:0/991857006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9881f38c0 con 0x7fb98806b970 2024-04-22T07:04:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.858+0000 7fb98e3d5640 1 -- 172.21.15.92:0/991857006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb988068b00 con 0x7fb98806b970 2024-04-22T07:04:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.859+0000 7fb984ff9640 1 -- 172.21.15.92:0/991857006 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb97c04b060 con 0x7fb98806b970 2024-04-22T07:04:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.859+0000 7fb984ff9640 1 --2- 172.21.15.92:0/991857006 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb97406c8f0 0x7fb97406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.859+0000 7fb984ff9640 1 -- 172.21.15.92:0/991857006 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb97c0ad6e0 con 0x7fb98806b970 2024-04-22T07:04:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.859+0000 7fb9877fe640 1 --2- 172.21.15.92:0/991857006 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb97406c8f0 0x7fb97406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.860+0000 7fb9877fe640 1 --2- 172.21.15.92:0/991857006 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb97406c8f0 0x7fb97406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb978006fd0 tx=0x7fb978008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:28.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:28.863+0000 7fb984ff9640 1 -- 172.21.15.92:0/991857006 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb97c02f300 con 0x7fb98806b970 2024-04-22T07:04:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.046+0000 7fb98e3d5640 1 -- 172.21.15.92:0/991857006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb98815ce60 con 0x7fb98806b970 2024-04-22T07:04:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.047+0000 7fb984ff9640 1 -- 172.21.15.92:0/991857006 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fb97c07ba80 con 0x7fb98806b970 2024-04-22T07:04:29.045 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:29.046 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:04:04.784969+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.049+0000 7fb98e3d5640 1 -- 172.21.15.92:0/991857006 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb97406c8f0 msgr2=0x7fb97406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.049+0000 7fb98e3d5640 1 --2- 172.21.15.92:0/991857006 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb97406c8f0 0x7fb97406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb978006fd0 tx=0x7fb978008040 comp rx=0 tx=0).stop 2024-04-22T07:04:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.049+0000 7fb98e3d5640 1 -- 172.21.15.92:0/991857006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb98806b970 msgr2=0x7fb98814c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.049+0000 7fb98e3d5640 1 --2- 172.21.15.92:0/991857006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb98806b970 0x7fb98814c320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb97c02b930 tx=0x7fb97c0046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.049+0000 7fb98e3d5640 1 -- 172.21.15.92:0/991857006 shutdown_connections 2024-04-22T07:04:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.049+0000 7fb98e3d5640 1 --2- 172.21.15.92:0/991857006 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fb97406c8f0 0x7fb97406edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.049+0000 7fb98e3d5640 1 --2- 172.21.15.92:0/991857006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb988152dd0 0x7fb9881f0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.049+0000 7fb98e3d5640 1 --2- 172.21.15.92:0/991857006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9880c9970 0x7fb9880c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.049+0000 7fb98e3d5640 1 --2- 172.21.15.92:0/991857006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb98806b970 0x7fb98814c320 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.049+0000 7fb98e3d5640 1 -- 172.21.15.92:0/991857006 >> 172.21.15.92:0/991857006 conn(0x7fb9880cd2d0 msgr2=0x7fb988151b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.050+0000 7fb98e3d5640 1 -- 172.21.15.92:0/991857006 shutdown_connections 2024-04-22T07:04:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.050+0000 7fb98e3d5640 1 -- 172.21.15.92:0/991857006 wait complete. 2024-04-22T07:04:29.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:29.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.159+0000 7f296ce14640 1 Processor -- start 2024-04-22T07:04:29.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.159+0000 7f296ce14640 1 -- start start 2024-04-22T07:04:29.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.159+0000 7f296ce14640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2968156190 0x7f2968070490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.159+0000 7f296ce14640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29680720e0 con 0x7f29680c43c0 2024-04-22T07:04:29.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.159+0000 7f296ce14640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2968072350 con 0x7f29680c8df0 2024-04-22T07:04:29.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.159+0000 7f296ce14640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f29680c3410 con 0x7f2968156190 2024-04-22T07:04:29.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.159+0000 7f2966d9d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f29680c43c0 0x7f2968163330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53048/0 (socket says 172.21.15.92:53048) 2024-04-22T07:04:29.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.159+0000 7f2966d9d640 1 -- 172.21.15.92:0/4231355396 learned_addr learned my addr 172.21.15.92:0/4231355396 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:29.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f296659c640 1 --2- 172.21.15.92:0/4231355396 >> v2:172.21.15.145:3300/0 conn(0x7f2968156190 0x7f2968070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2738503854 0 0) 0x7f29680720e0 con 0x7f29680c43c0 2024-04-22T07:04:29.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2954003520 con 0x7f29680c43c0 2024-04-22T07:04:29.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4072343889 0 0) 0x7f2954003520 con 0x7f29680c43c0 2024-04-22T07:04:29.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 >> v2:172.21.15.145:3300/0 conn(0x7f2968156190 msgr2=0x7f2968070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:29.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 --2- 172.21.15.92:0/4231355396 >> v2:172.21.15.145:3300/0 conn(0x7f2968156190 0x7f2968070490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 >> v1:172.21.15.145:6789/0 conn(0x7f29680c8df0 legacy=0x7f29680c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29680c3660 con 0x7f29680c43c0 2024-04-22T07:04:29.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f2960001a10 con 0x7f29680c43c0 2024-04-22T07:04:29.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f296659c640 1 --2- 172.21.15.92:0/4231355396 >> v2:172.21.15.145:3300/0 conn(0x7f2968156190 0x7f2968070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:29.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 >> v1:172.21.15.92:6789/0 conn(0x7f29680c43c0 legacy=0x7f2968163330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 --2- 172.21.15.92:0/4231355396 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29540039b0 0x7f2954003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 --2- 172.21.15.92:0/4231355396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2954004800 0x7f2954004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 --2- 172.21.15.92:0/4231355396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29540051c0 0x7f295400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f29680c3410 con 0x7f29540051c0 2024-04-22T07:04:29.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f295400c060 con 0x7f2954004800 2024-04-22T07:04:29.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f295400c260 con 0x7f29540039b0 2024-04-22T07:04:29.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2966d9d640 1 --2- 172.21.15.92:0/4231355396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29540051c0 0x7f295400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f296659c640 1 --2- 172.21.15.92:0/4231355396 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29540039b0 0x7f2954003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f295ffff640 1 --2- 172.21.15.92:0/4231355396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2954004800 0x7f2954004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2966d9d640 1 -- 172.21.15.92:0/4231355396 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29540039b0 msgr2=0x7f2954003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:29.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2966d9d640 1 --2- 172.21.15.92:0/4231355396 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29540039b0 0x7f2954003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.163 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2966d9d640 1 -- 172.21.15.92:0/4231355396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2954004800 msgr2=0x7f2954004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:29.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2966d9d640 1 --2- 172.21.15.92:0/4231355396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2954004800 0x7f2954004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.160+0000 7f2966d9d640 1 -- 172.21.15.92:0/4231355396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2954003520 con 0x7f29540051c0 2024-04-22T07:04:29.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.161+0000 7f2966d9d640 1 -- 172.21.15.92:0/4231355396 reap_dead start 2024-04-22T07:04:29.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.161+0000 7f2966d9d640 1 --2- 172.21.15.92:0/4231355396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29540051c0 0x7f295400b9c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f29680722b0 tx=0x7f2954005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204eb8649f1d5efd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:29.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.161+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296000a480 con 0x7f29540051c0 2024-04-22T07:04:29.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.161+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2960034650 con 0x7f29540051c0 2024-04-22T07:04:29.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.161+0000 7f2965d9b640 1 -- 172.21.15.92:0/4231355396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296002ce60 con 0x7f29540051c0 2024-04-22T07:04:29.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.162+0000 7f296ce14640 1 -- 172.21.15.92:0/4231355396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29540051c0 msgr2=0x7f295400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.162+0000 7f296ce14640 1 --2- 172.21.15.92:0/4231355396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29540051c0 0x7f295400b9c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f29680722b0 tx=0x7f2954005830 comp rx=0 tx=0).stop 2024-04-22T07:04:29.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.162+0000 7f296ce14640 1 -- 172.21.15.92:0/4231355396 shutdown_connections 2024-04-22T07:04:29.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.162+0000 7f296ce14640 1 --2- 172.21.15.92:0/4231355396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29540051c0 0x7f295400b9c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.162+0000 7f296ce14640 1 -- 172.21.15.92:0/4231355396 >> 172.21.15.92:0/4231355396 conn(0x7f296806e260 msgr2=0x7f29680715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:29.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.162+0000 7f296ce14640 1 -- 172.21.15.92:0/4231355396 shutdown_connections 2024-04-22T07:04:29.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.162+0000 7f296ce14640 1 -- 172.21.15.92:0/4231355396 wait complete. 2024-04-22T07:04:29.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.163+0000 7f296ce14640 1 Processor -- start 2024-04-22T07:04:29.165 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.163+0000 7f296ce14640 1 -- start start 2024-04-22T07:04:29.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f296ce14640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29540051c0 0x7f29681ef290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f296ce14640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f296814f990 0x7f2968147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f296ce14640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29681551b0 0x7f2968155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f296ce14640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f296806f9b0 con 0x7f29681551b0 2024-04-22T07:04:29.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f296ce14640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2968070000 con 0x7f29540051c0 2024-04-22T07:04:29.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f296ce14640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f296806fc90 con 0x7f296814f990 2024-04-22T07:04:29.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f2966d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29681551b0 0x7f2968155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f2966d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29681551b0 0x7f2968155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39182/0 (socket says 172.21.15.92:39182) 2024-04-22T07:04:29.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f2966d9d640 1 -- 172.21.15.92:0/9908216 learned_addr learned my addr 172.21.15.92:0/9908216 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:29.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f295ffff640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f296814f990 0x7f2968147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f296659c640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29540051c0 0x7f29681ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f2966d9d640 1 -- 172.21.15.92:0/9908216 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f296814f990 msgr2=0x7f2968147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f2966d9d640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f296814f990 0x7f2968147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f2966d9d640 1 -- 172.21.15.92:0/9908216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29540051c0 msgr2=0x7f29681ef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f2966d9d640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29540051c0 0x7f29681ef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f2966d9d640 1 -- 172.21.15.92:0/9908216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296814cf20 con 0x7f29681551b0 2024-04-22T07:04:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.164+0000 7f2966d9d640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29681551b0 0x7f2968155630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f296002cbf0 tx=0x7f2968071f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.165+0000 7f295effd640 1 -- 172.21.15.92:0/9908216 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960039040 con 0x7f29681551b0 2024-04-22T07:04:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.165+0000 7f295effd640 1 -- 172.21.15.92:0/9908216 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2960043070 con 0x7f29681551b0 2024-04-22T07:04:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.165+0000 7f295effd640 1 -- 172.21.15.92:0/9908216 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960001a10 con 0x7f29681551b0 2024-04-22T07:04:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.165+0000 7f296ce14640 1 -- 172.21.15.92:0/9908216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29681eead0 con 0x7f29681551b0 2024-04-22T07:04:29.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.165+0000 7f296ce14640 1 -- 172.21.15.92:0/9908216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296814da90 con 0x7f29681551b0 2024-04-22T07:04:29.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.166+0000 7f296ce14640 1 -- 172.21.15.92:0/9908216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2968068b00 con 0x7f29681551b0 2024-04-22T07:04:29.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.167+0000 7f295effd640 1 -- 172.21.15.92:0/9908216 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2960034650 con 0x7f29681551b0 2024-04-22T07:04:29.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.167+0000 7f295effd640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f294806c8f0 0x7f294806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.167+0000 7f296659c640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f294806c8f0 0x7f294806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.167+0000 7f295effd640 1 -- 172.21.15.92:0/9908216 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f29600ad750 con 0x7f29681551b0 2024-04-22T07:04:29.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.167+0000 7f296659c640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f294806c8f0 0x7f294806edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29580093d0 tx=0x7f2958009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:29.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.171+0000 7f295effd640 1 -- 172.21.15.92:0/9908216 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f296007b530 con 0x7f29681551b0 2024-04-22T07:04:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.390+0000 7f296ce14640 1 -- 172.21.15.92:0/9908216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29680b43e0 con 0x7f29681551b0 2024-04-22T07:04:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.390+0000 7f295effd640 1 -- 172.21.15.92:0/9908216 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6909 (secure 0 0 0) 0x7f296007f830 con 0x7f29681551b0 2024-04-22T07:04:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:29.389 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1059009649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1059009649},{"type":"v1","addr":"172.21.15.145:6835","nonce":1059009649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:03.129095+0000","modified":"2024-04-22T07:04:25.017461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4275,"mds_1":4273,"mds_2":4269,"mds_3":4232,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.145:6836/1058270486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1058270486},{"type":"v1","addr":"172.21.15.145:6836","nonce":1058270486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4232":{"gid":4232,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.145:6837/3258302920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3258302920},{"type":"v1","addr":"172.21.15.145:6837","nonce":3258302920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/101798872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":101798872},{"type":"v1","addr":"172.21.15.92:6837","nonce":101798872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6839/3493022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3493022212},{"type":"v1","addr":"172.21.15.92:6839","nonce":3493022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275,4273,4269,4232]},"id":1}]} 2024-04-22T07:04:29.389 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 16 2024-04-22T07:04:29.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.392+0000 7f296ce14640 1 -- 172.21.15.92:0/9908216 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f294806c8f0 msgr2=0x7f294806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.392+0000 7f296ce14640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f294806c8f0 0x7f294806edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29580093d0 tx=0x7f2958009190 comp rx=0 tx=0).stop 2024-04-22T07:04:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.392+0000 7f296ce14640 1 -- 172.21.15.92:0/9908216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29681551b0 msgr2=0x7f2968155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.392+0000 7f296ce14640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29681551b0 0x7f2968155630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f296002cbf0 tx=0x7f2968071f90 comp rx=0 tx=0).stop 2024-04-22T07:04:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.392+0000 7f296ce14640 1 -- 172.21.15.92:0/9908216 shutdown_connections 2024-04-22T07:04:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.392+0000 7f296ce14640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f294806c8f0 0x7f294806edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.392+0000 7f296ce14640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29681551b0 0x7f2968155630 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.392+0000 7f296ce14640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f296814f990 0x7f2968147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.392+0000 7f296ce14640 1 --2- 172.21.15.92:0/9908216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29540051c0 0x7f29681ef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.392+0000 7f296ce14640 1 -- 172.21.15.92:0/9908216 >> 172.21.15.92:0/9908216 conn(0x7f296806e260 msgr2=0x7f29681543e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.392+0000 7f296ce14640 1 -- 172.21.15.92:0/9908216 shutdown_connections 2024-04-22T07:04:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:29.392+0000 7f296ce14640 1 -- 172.21.15.92:0/9908216 wait complete. 2024-04-22T07:04:30.007 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-22T07:04:03.129095+0000', 'modified': '2024-04-22T07:04:25.017461+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4275, 'mds_1': 4273, 'mds_2': 4269, 'mds_3': 4232, 'mds_4': 4229}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4229': {'gid': 4229, 'name': 'b', 'rank': 4, 'incarnation': 16, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.145:6836/1058270486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6833', 'nonce': 1058270486}, {'type': 'v1', 'addr': '172.21.15.145:6836', 'nonce': 1058270486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4232': {'gid': 4232, 'name': 'f', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.145:6837/3258302920', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 3258302920}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 3258302920}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4269': {'gid': 4269, 'name': 'c', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.92:6836/1444537065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 1444537065}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 1444537065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4273': {'gid': 4273, 'name': 'a', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6837/101798872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 101798872}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 101798872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4275': {'gid': 4275, 'name': 'e', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.92:6839/3493022212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 3493022212}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 3493022212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4275, 'qdb_cluster': [4275, 4273, 4269, 4232]} 2024-04-22T07:04:30.007 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4229:up:creating 2024-04-22T07:04:31.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:31.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.111+0000 7f6e8b775640 1 Processor -- start 2024-04-22T07:04:31.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.111+0000 7f6e8b775640 1 -- start start 2024-04-22T07:04:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e8b775640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6e84152c90 0x7f6e840c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e8b775640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e84159280 0x7f6e840c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e8b775640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e84067f10 con 0x7f6e84153070 2024-04-22T07:04:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e8b775640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f6e8406bd90 con 0x7f6e84152c90 2024-04-22T07:04:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e8b775640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6e840c7e50 con 0x7f6e84159280 2024-04-22T07:04:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e89511640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6e84152c90 0x7f6e840c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e89511640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6e84152c90 0x7f6e840c6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39186/0 (socket says 172.21.15.92:39186) 2024-04-22T07:04:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e88d10640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6e84153070 0x7f6e84158b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53050/0 (socket says 172.21.15.92:53050) 2024-04-22T07:04:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e89511640 1 -- 172.21.15.92:0/3273439899 learned_addr learned my addr 172.21.15.92:0/3273439899 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e89d12640 1 --2- 172.21.15.92:0/3273439899 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e84159280 0x7f6e840c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e89511640 1 -- 172.21.15.92:0/3273439899 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e84159280 msgr2=0x7f6e840c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e89511640 1 --2- 172.21.15.92:0/3273439899 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e84159280 0x7f6e840c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e89511640 1 -- 172.21.15.92:0/3273439899 >> v1:172.21.15.92:6789/0 conn(0x7f6e84153070 legacy=0x7f6e84158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.112+0000 7f6e89511640 1 -- 172.21.15.92:0/3273439899 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e840c8110 con 0x7f6e84152c90 2024-04-22T07:04:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.113+0000 7f6e89511640 1 --2- 172.21.15.92:0/3273439899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e84152c90 0x7f6e840c6a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009870 tx=0x7f6e7402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71c824fc4f00844 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.113+0000 7f6e7bfff640 1 -- 172.21.15.92:0/3273439899 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74034070 con 0x7f6e84152c90 2024-04-22T07:04:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.113+0000 7f6e7bfff640 1 -- 172.21.15.92:0/3273439899 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e74030840 con 0x7f6e84152c90 2024-04-22T07:04:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.113+0000 7f6e7bfff640 1 -- 172.21.15.92:0/3273439899 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74030c10 con 0x7f6e84152c90 2024-04-22T07:04:31.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.114+0000 7f6e8b775640 1 -- 172.21.15.92:0/3273439899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e84152c90 msgr2=0x7f6e840c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:31.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.114+0000 7f6e8b775640 1 --2- 172.21.15.92:0/3273439899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e84152c90 0x7f6e840c6a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009870 tx=0x7f6e7402b980 comp rx=0 tx=0).stop 2024-04-22T07:04:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.114+0000 7f6e8b775640 1 -- 172.21.15.92:0/3273439899 shutdown_connections 2024-04-22T07:04:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.114+0000 7f6e8b775640 1 --2- 172.21.15.92:0/3273439899 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e84159280 0x7f6e840c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.114+0000 7f6e8b775640 1 --2- 172.21.15.92:0/3273439899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e84152c90 0x7f6e840c6a50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.114+0000 7f6e8b775640 1 -- 172.21.15.92:0/3273439899 >> 172.21.15.92:0/3273439899 conn(0x7f6e84072ee0 msgr2=0x7f6e8406e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.114+0000 7f6e8b775640 1 -- 172.21.15.92:0/3273439899 shutdown_connections 2024-04-22T07:04:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.114+0000 7f6e8b775640 1 -- 172.21.15.92:0/3273439899 wait complete. 2024-04-22T07:04:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.115+0000 7f6e8b775640 1 Processor -- start 2024-04-22T07:04:31.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.116+0000 7f6e8b775640 1 -- start start 2024-04-22T07:04:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e8b775640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e84152c90 0x7f6e841f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:31.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e8b775640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e84153070 0x7f6e8414f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e8b775640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e84159280 0x7f6e8414f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e8b775640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6e84067830 con 0x7f6e84152c90 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e8b775640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6e840676b0 con 0x7f6e84159280 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e8b775640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6e840c83f0 con 0x7f6e84153070 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e89511640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e84152c90 0x7f6e841f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e89511640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e84152c90 0x7f6e841f1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39192/0 (socket says 172.21.15.92:39192) 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e89511640 1 -- 172.21.15.92:0/2071378167 learned_addr learned my addr 172.21.15.92:0/2071378167 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e89511640 1 -- 172.21.15.92:0/2071378167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e84153070 msgr2=0x7f6e8414f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e89511640 1 --2- 172.21.15.92:0/2071378167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e84153070 0x7f6e8414f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e89511640 1 -- 172.21.15.92:0/2071378167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e84159280 msgr2=0x7f6e8414f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e89511640 1 --2- 172.21.15.92:0/2071378167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e84159280 0x7f6e8414f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e89511640 1 -- 172.21.15.92:0/2071378167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e84158960 con 0x7f6e84152c90 2024-04-22T07:04:31.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e89511640 1 --2- 172.21.15.92:0/2071378167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e84152c90 0x7f6e841f1aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6e74000c00 tx=0x7f6e74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:31.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e79ffb640 1 -- 172.21.15.92:0/2071378167 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74048070 con 0x7f6e84152c90 2024-04-22T07:04:31.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e79ffb640 1 -- 172.21.15.92:0/2071378167 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e74030640 con 0x7f6e84152c90 2024-04-22T07:04:31.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e8b775640 1 -- 172.21.15.92:0/2071378167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e841574b0 con 0x7f6e84152c90 2024-04-22T07:04:31.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.117+0000 7f6e79ffb640 1 -- 172.21.15.92:0/2071378167 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74034040 con 0x7f6e84152c90 2024-04-22T07:04:31.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.118+0000 7f6e8b775640 1 -- 172.21.15.92:0/2071378167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e841f1fe0 con 0x7f6e84152c90 2024-04-22T07:04:31.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.119+0000 7f6e8b775640 1 -- 172.21.15.92:0/2071378167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e84069930 con 0x7f6e84152c90 2024-04-22T07:04:31.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.119+0000 7f6e79ffb640 1 -- 172.21.15.92:0/2071378167 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e7404b070 con 0x7f6e84152c90 2024-04-22T07:04:31.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.120+0000 7f6e79ffb640 1 --2- 172.21.15.92:0/2071378167 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f6e6806c8f0 0x7f6e6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:31.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.120+0000 7f6e88d10640 1 --2- 172.21.15.92:0/2071378167 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f6e6806c8f0 0x7f6e6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:31.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.120+0000 7f6e79ffb640 1 -- 172.21.15.92:0/2071378167 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6e740adb10 con 0x7f6e84152c90 2024-04-22T07:04:31.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.120+0000 7f6e88d10640 1 --2- 172.21.15.92:0/2071378167 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f6e6806c8f0 0x7f6e6806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6e80000f60 tx=0x7f6e80007a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:31.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.124+0000 7f6e79ffb640 1 -- 172.21.15.92:0/2071378167 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e84069930 con 0x7f6e84152c90 2024-04-22T07:04:31.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.337+0000 7f6e8b775640 1 -- 172.21.15.92:0/2071378167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e8414d750 con 0x7f6e84152c90 2024-04-22T07:04:31.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.337+0000 7f6e79ffb640 1 -- 172.21.15.92:0/2071378167 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6913 (secure 0 0 0) 0x7f6e8414d750 con 0x7f6e84152c90 2024-04-22T07:04:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:31.336 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1059009649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1059009649},{"type":"v1","addr":"172.21.15.145:6835","nonce":1059009649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:03.129095+0000","modified":"2024-04-22T07:04:31.038967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4275,"mds_1":4273,"mds_2":4269,"mds_3":4232,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.145:6836/1058270486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1058270486},{"type":"v1","addr":"172.21.15.145:6836","nonce":1058270486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4232":{"gid":4232,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.145:6837/3258302920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3258302920},{"type":"v1","addr":"172.21.15.145:6837","nonce":3258302920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/101798872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":101798872},{"type":"v1","addr":"172.21.15.92:6837","nonce":101798872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6839/3493022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3493022212},{"type":"v1","addr":"172.21.15.92:6839","nonce":3493022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4229,4232,4269,4273,4275]},"id":1}]} 2024-04-22T07:04:31.336 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2024-04-22T07:04:31.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.339+0000 7f6e8b775640 1 -- 172.21.15.92:0/2071378167 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f6e6806c8f0 msgr2=0x7f6e6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:31.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.339+0000 7f6e8b775640 1 --2- 172.21.15.92:0/2071378167 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f6e6806c8f0 0x7f6e6806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6e80000f60 tx=0x7f6e80007a00 comp rx=0 tx=0).stop 2024-04-22T07:04:31.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.339+0000 7f6e8b775640 1 -- 172.21.15.92:0/2071378167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e84152c90 msgr2=0x7f6e841f1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:31.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.339+0000 7f6e8b775640 1 --2- 172.21.15.92:0/2071378167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e84152c90 0x7f6e841f1aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6e74000c00 tx=0x7f6e74004730 comp rx=0 tx=0).stop 2024-04-22T07:04:31.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.339+0000 7f6e8b775640 1 -- 172.21.15.92:0/2071378167 shutdown_connections 2024-04-22T07:04:31.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.339+0000 7f6e8b775640 1 --2- 172.21.15.92:0/2071378167 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f6e6806c8f0 0x7f6e6806edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:31.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.339+0000 7f6e8b775640 1 --2- 172.21.15.92:0/2071378167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e84159280 0x7f6e8414f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:31.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.339+0000 7f6e8b775640 1 --2- 172.21.15.92:0/2071378167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e84153070 0x7f6e8414f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:31.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.339+0000 7f6e8b775640 1 --2- 172.21.15.92:0/2071378167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e84152c90 0x7f6e841f1aa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:31.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.339+0000 7f6e8b775640 1 -- 172.21.15.92:0/2071378167 >> 172.21.15.92:0/2071378167 conn(0x7f6e84072ee0 msgr2=0x7f6e84151480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:31.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.339+0000 7f6e8b775640 1 -- 172.21.15.92:0/2071378167 shutdown_connections 2024-04-22T07:04:31.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:31.339+0000 7f6e8b775640 1 -- 172.21.15.92:0/2071378167 wait complete. 2024-04-22T07:04:31.952 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-22T07:04:03.129095+0000', 'modified': '2024-04-22T07:04:31.038967+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4275, 'mds_1': 4273, 'mds_2': 4269, 'mds_3': 4232, 'mds_4': 4229}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4229': {'gid': 4229, 'name': 'b', 'rank': 4, 'incarnation': 16, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.145:6836/1058270486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6833', 'nonce': 1058270486}, {'type': 'v1', 'addr': '172.21.15.145:6836', 'nonce': 1058270486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4232': {'gid': 4232, 'name': 'f', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.145:6837/3258302920', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 3258302920}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 3258302920}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4269': {'gid': 4269, 'name': 'c', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.92:6836/1444537065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 1444537065}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 1444537065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4273': {'gid': 4273, 'name': 'a', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6837/101798872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 101798872}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 101798872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4275': {'gid': 4275, 'name': 'e', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.92:6839/3493022212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 3493022212}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 3493022212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4275, 'qdb_cluster': [4229, 4232, 4269, 4273, 4275]} 2024-04-22T07:04:31.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-22T07:04:31.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T07:04:32.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.054+0000 7f101ab7e640 1 Processor -- start 2024-04-22T07:04:32.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.054+0000 7f101ab7e640 1 -- start start 2024-04-22T07:04:32.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f101ab7e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1014151d10 0x7f1014073820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f101ab7e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10140c8dc0 0x7f10140c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f101ab7e640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10140c69d0 0x7f1014157b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f101ab7e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f101406d2d0 con 0x7f10140c8dc0 2024-04-22T07:04:32.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f101ab7e640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1014072920 con 0x7f10140c69d0 2024-04-22T07:04:32.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f101ab7e640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f10140c47d0 con 0x7f1014151d10 2024-04-22T07:04:32.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f1013fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10140c8dc0 0x7f10140c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:32.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f1013fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10140c8dc0 0x7f10140c6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39206/0 (socket says 172.21.15.92:39206) 2024-04-22T07:04:32.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f1013fff640 1 -- 172.21.15.92:0/3069683953 learned_addr learned my addr 172.21.15.92:0/3069683953 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:32.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f1013fff640 1 -- 172.21.15.92:0/3069683953 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1014151d10 msgr2=0x7f1014073820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:32.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f101891a640 1 --2- 172.21.15.92:0/3069683953 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1014151d10 0x7f1014073820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:32.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f101911b640 1 --2- 172.21.15.92:0/3069683953 >> v2:172.21.15.145:3300/0 conn(0x7f10140c69d0 0x7f1014157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:32.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f1013fff640 1 --2- 172.21.15.92:0/3069683953 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1014151d10 0x7f1014073820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f1013fff640 1 -- 172.21.15.92:0/3069683953 >> v2:172.21.15.145:3300/0 conn(0x7f10140c69d0 msgr2=0x7f1014157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:32.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f1013fff640 1 --2- 172.21.15.92:0/3069683953 >> v2:172.21.15.145:3300/0 conn(0x7f10140c69d0 0x7f1014157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.055+0000 7f1013fff640 1 -- 172.21.15.92:0/3069683953 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10141520f0 con 0x7f10140c8dc0 2024-04-22T07:04:32.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.056+0000 7f1013fff640 1 --2- 172.21.15.92:0/3069683953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10140c8dc0 0x7f10140c6490 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f10140c7970 tx=0x7f100002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5581107b42bf5481 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:32.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.056+0000 7f10137fe640 1 -- 172.21.15.92:0/3069683953 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100003b070 con 0x7f10140c8dc0 2024-04-22T07:04:32.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.056+0000 7f10137fe640 1 -- 172.21.15.92:0/3069683953 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1000036440 con 0x7f10140c8dc0 2024-04-22T07:04:32.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.056+0000 7f10137fe640 1 -- 172.21.15.92:0/3069683953 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000036810 con 0x7f10140c8dc0 2024-04-22T07:04:32.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.056+0000 7f101ab7e640 1 -- 172.21.15.92:0/3069683953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10140c8dc0 msgr2=0x7f10140c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:32.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.056+0000 7f101ab7e640 1 --2- 172.21.15.92:0/3069683953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10140c8dc0 0x7f10140c6490 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f10140c7970 tx=0x7f100002e0e0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.056+0000 7f101ab7e640 1 -- 172.21.15.92:0/3069683953 shutdown_connections 2024-04-22T07:04:32.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.056+0000 7f101ab7e640 1 --2- 172.21.15.92:0/3069683953 >> v2:172.21.15.145:3300/0 conn(0x7f10140c69d0 0x7f1014157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.056+0000 7f101ab7e640 1 --2- 172.21.15.92:0/3069683953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10140c8dc0 0x7f10140c6490 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.056+0000 7f101ab7e640 1 --2- 172.21.15.92:0/3069683953 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1014151d10 0x7f1014073820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.056+0000 7f101ab7e640 1 -- 172.21.15.92:0/3069683953 >> 172.21.15.92:0/3069683953 conn(0x7f10140c9dd0 msgr2=0x7f10140ca1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:32.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.057+0000 7f101ab7e640 1 -- 172.21.15.92:0/3069683953 shutdown_connections 2024-04-22T07:04:32.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.057+0000 7f101ab7e640 1 -- 172.21.15.92:0/3069683953 wait complete. 2024-04-22T07:04:32.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.058+0000 7f101ab7e640 1 Processor -- start 2024-04-22T07:04:32.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.058+0000 7f101ab7e640 1 -- start start 2024-04-22T07:04:32.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101ab7e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10140c69d0 0x7f10141ef940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101ab7e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10140c8dc0 0x7f10141ecd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101ab7e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1014151d10 0x7f101414f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101ab7e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1014072bc0 con 0x7f10140c69d0 2024-04-22T07:04:32.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101ab7e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10140c4d70 con 0x7f1014151d10 2024-04-22T07:04:32.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101ab7e640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1014072ea0 con 0x7f10140c8dc0 2024-04-22T07:04:32.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101891a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10140c69d0 0x7f10141ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101891a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10140c69d0 0x7f10141ef940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39210/0 (socket says 172.21.15.92:39210) 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101891a640 1 -- 172.21.15.92:0/164652441 learned_addr learned my addr 172.21.15.92:0/164652441 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f1013fff640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10140c8dc0 0x7f10141ecd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101911b640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1014151d10 0x7f101414f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101891a640 1 -- 172.21.15.92:0/164652441 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10140c8dc0 msgr2=0x7f10141ecd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101891a640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10140c8dc0 0x7f10141ecd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101891a640 1 -- 172.21.15.92:0/164652441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1014151d10 msgr2=0x7f101414f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101891a640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1014151d10 0x7f101414f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101891a640 1 -- 172.21.15.92:0/164652441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1014157920 con 0x7f10140c69d0 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101891a640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10140c69d0 0x7f10141ef940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f100800c7f0 tx=0x7f100800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f10117fa640 1 -- 172.21.15.92:0/164652441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008015070 con 0x7f10140c69d0 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f10117fa640 1 -- 172.21.15.92:0/164652441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1008004220 con 0x7f10140c69d0 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f10117fa640 1 -- 172.21.15.92:0/164652441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008003610 con 0x7f10140c69d0 2024-04-22T07:04:32.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101ab7e640 1 -- 172.21.15.92:0/164652441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10141f0820 con 0x7f10140c69d0 2024-04-22T07:04:32.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.060+0000 7f101ab7e640 1 -- 172.21.15.92:0/164652441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10141f0260 con 0x7f10140c69d0 2024-04-22T07:04:32.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.062+0000 7f10117fa640 1 -- 172.21.15.92:0/164652441 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10080039c0 con 0x7f10140c69d0 2024-04-22T07:04:32.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.062+0000 7f10117fa640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0ff806c920 0x7f0ff806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.063+0000 7f1013fff640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0ff806c920 0x7f0ff806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:32.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.063+0000 7f10117fa640 1 -- 172.21.15.92:0/164652441 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f100808baf0 con 0x7f10140c69d0 2024-04-22T07:04:32.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.063+0000 7f1013fff640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0ff806c920 0x7f0ff806ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1000038000 tx=0x7f1000032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:32.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.063+0000 7f101d76d740 1 Processor -- start 2024-04-22T07:04:32.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.063+0000 7f101d76d740 1 -- start start 2024-04-22T07:04:32.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101d76d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef8ee40 0x55e4cef97f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101d76d740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef4fec0 0x55e4cef984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101d76d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4bca0 0x55e4cef73ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101d76d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e4ceeb2b10 con 0x55e4cef4bca0 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101d76d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e4cee50240 con 0x55e4cef8ee40 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101d76d740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55e4ceeaead0 con 0x55e4cef4fec0 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101911b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4bca0 0x55e4cef73ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101911b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4bca0 0x55e4cef73ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39212/0 (socket says 172.21.15.92:39212) 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101911b640 1 -- 172.21.15.92:0/3735014764 learned_addr learned my addr 172.21.15.92:0/3735014764 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f1013fff640 1 --2- 172.21.15.92:0/3735014764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef4fec0 0x55e4cef984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101911b640 1 -- 172.21.15.92:0/3735014764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef4fec0 msgr2=0x55e4cef984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101911b640 1 --2- 172.21.15.92:0/3735014764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef4fec0 0x55e4cef984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101911b640 1 -- 172.21.15.92:0/3735014764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef8ee40 msgr2=0x55e4cef97f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101911b640 1 --2- 172.21.15.92:0/3735014764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef8ee40 0x55e4cef97f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101911b640 1 -- 172.21.15.92:0/3735014764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4cebc52d0 con 0x55e4cef4bca0 2024-04-22T07:04:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.064+0000 7f101911b640 1 --2- 172.21.15.92:0/3735014764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4bca0 0x55e4cef73ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f100c007fd0 tx=0x7f100c00f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86dcfe98125605d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.065+0000 7f0ffe7fc640 1 -- 172.21.15.92:0/3735014764 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c01c070 con 0x55e4cef4bca0 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.065+0000 7f0ffe7fc640 1 -- 172.21.15.92:0/3735014764 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100c00d710 con 0x55e4cef4bca0 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.065+0000 7f0ffe7fc640 1 -- 172.21.15.92:0/3735014764 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c00dac0 con 0x55e4cef4bca0 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.065+0000 7f101d76d740 1 -- 172.21.15.92:0/3735014764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4bca0 msgr2=0x55e4cef73ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.065+0000 7f101d76d740 1 --2- 172.21.15.92:0/3735014764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4bca0 0x55e4cef73ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f100c007fd0 tx=0x7f100c00f910 comp rx=0 tx=0).stop 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.065+0000 7f101d76d740 1 -- 172.21.15.92:0/3735014764 shutdown_connections 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.065+0000 7f101d76d740 1 --2- 172.21.15.92:0/3735014764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4bca0 0x55e4cef73ee0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.065+0000 7f101d76d740 1 --2- 172.21.15.92:0/3735014764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef4fec0 0x55e4cef984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.065+0000 7f101d76d740 1 --2- 172.21.15.92:0/3735014764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef8ee40 0x55e4cef97f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.065+0000 7f101d76d740 1 -- 172.21.15.92:0/3735014764 >> 172.21.15.92:0/3735014764 conn(0x55e4cef4a990 msgr2=0x55e4cef4d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.065+0000 7f101d76d740 1 -- 172.21.15.92:0/3735014764 shutdown_connections 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.065+0000 7f101d76d740 1 -- 172.21.15.92:0/3735014764 wait complete. 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.066+0000 7f101d76d740 1 -- start start 2024-04-22T07:04:32.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.066+0000 7f101d76d740 1 Processor -- start 2024-04-22T07:04:32.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101d76d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4fec0 0x55e4cef95d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101d76d740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef8ee40 0x55e4cef6d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101d76d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef74490 0x55e4cef885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101d76d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e4cef4e250 con 0x55e4cef4fec0 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101d76d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e4cee3b190 con 0x55e4cef74490 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101d76d740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55e4cef4e530 con 0x55e4cef8ee40 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101d76d740 10 client.0 _ll_register_callbacks cb 0x55e4cee368b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101891a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4fec0 0x55e4cef95d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101891a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4fec0 0x55e4cef95d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39218/0 (socket says 172.21.15.92:39218) 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101891a640 1 -- 172.21.15.92:0/1287143223 learned_addr learned my addr 172.21.15.92:0/1287143223 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101891a640 1 -- 172.21.15.92:0/1287143223 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef8ee40 msgr2=0x55e4cef6d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101891a640 1 --2- 172.21.15.92:0/1287143223 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef8ee40 0x55e4cef6d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101891a640 1 -- 172.21.15.92:0/1287143223 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef74490 msgr2=0x55e4cef885e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101891a640 1 --2- 172.21.15.92:0/1287143223 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef74490 0x55e4cef885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101891a640 1 -- 172.21.15.92:0/1287143223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4cec8f6b0 con 0x55e4cef4fec0 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101891a640 1 --2- 172.21.15.92:0/1287143223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4fec0 0x55e4cef95d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1008061ed0 tx=0x7f1008059290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28033a49b7c32216 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f0ff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f0ff37fe640 1 -- 172.21.15.92:0/1287143223 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008088060 con 0x55e4cef4fec0 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f0ff37fe640 1 -- 172.21.15.92:0/1287143223 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1008058e90 con 0x55e4cef4fec0 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101d76d740 1 -- 172.21.15.92:0/1287143223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f100c00d710 con 0x55e4cef4fec0 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f0ff37fe640 1 -- 172.21.15.92:0/1287143223 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f1008076050 con 0x55e4cef4fec0 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101d76d740 10 client.4734 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.067+0000 7f101d76d740 1 -- 172.21.15.92:0/1287143223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55e4cef95300 con 0x55e4cef4fec0 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.068+0000 7f0ff37fe640 1 -- 172.21.15.92:0/1287143223 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100c00d710 con 0x55e4cef4fec0 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.068+0000 7f0ff37fe640 1 -- 172.21.15.92:0/1287143223 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f1008055590 con 0x55e4cef4fec0 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.068+0000 7f101d76d740 10 client.4734 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.068+0000 7f101d76d740 10 client.4734 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.068+0000 7f101d76d740 20 client.4734 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.068+0000 7f101d76d740 1 --2- 172.21.15.92:0/1287143223 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55e4cef939a0 0x55e4cefaec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.068+0000 7f101d76d740 4 client.4734 mds_command: new command op to 4229 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:32.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.068+0000 7f101d76d740 1 -- 172.21.15.92:0/1287143223 --> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4ceec0530 con 0x55e4cef939a0 2024-04-22T07:04:32.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.068+0000 7f1013fff640 1 --2- 172.21.15.92:0/1287143223 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55e4cef939a0 0x55e4cefaec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:32.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.068+0000 7f1013fff640 1 --2- 172.21.15.92:0/1287143223 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55e4cef939a0 0x55e4cefaec70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a48990d474e350e4 server_cookie=3cfc36d030f54ba0 in_seq=0 out_seq=0 2024-04-22T07:04:32.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:32.068+0000 7f0ff37fe640 10 client.4734 ms_handle_connect on v2:172.21.15.145:6833/1058270486 2024-04-22T07:04:32.912 INFO:tasks.ceph.mds.e.smithi092.stderr:2024-04-22T07:04:32.914+0000 7ff94f4d7640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-22T07:04:35.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:04:35.016 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:04:36.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f0ff37fe640 1 -- 172.21.15.92:0/1287143223 <== mds.4 v2:172.21.15.145:6833/1058270486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4ceec0530 con 0x55e4cef939a0 2024-04-22T07:04:36.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f0ff37fe640 10 client.4734 handle_command_reply: tid=0 2024-04-22T07:04:36.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f101d76d740 1 client.4734 shutdown 2024-04-22T07:04:36.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f101d76d740 1 -- 172.21.15.92:0/1287143223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4fec0 msgr2=0x55e4cef95d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f101d76d740 1 --2- 172.21.15.92:0/1287143223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4fec0 0x55e4cef95d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1008061ed0 tx=0x7f1008059290 comp rx=0 tx=0).stop 2024-04-22T07:04:36.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f101d76d740 1 -- 172.21.15.92:0/1287143223 shutdown_connections 2024-04-22T07:04:36.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f101d76d740 1 --2- 172.21.15.92:0/1287143223 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55e4cef939a0 0x55e4cefaec70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f101d76d740 1 --2- 172.21.15.92:0/1287143223 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef74490 0x55e4cef885e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f0ff37fe640 0 client.4734 ms_handle_reset on v2:172.21.15.145:6833/1058270486 2024-04-22T07:04:36.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f101d76d740 1 --2- 172.21.15.92:0/1287143223 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef8ee40 0x55e4cef6d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f101d76d740 1 --2- 172.21.15.92:0/1287143223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef4fec0 0x55e4cef95d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f101d76d740 1 -- 172.21.15.92:0/1287143223 >> 172.21.15.92:0/1287143223 conn(0x55e4cef4bca0 msgr2=0x55e4cef6c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f101d76d740 1 -- 172.21.15.92:0/1287143223 shutdown_connections 2024-04-22T07:04:36.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f101d76d740 1 -- 172.21.15.92:0/1287143223 wait complete. 2024-04-22T07:04:36.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.042+0000 7f101d76d740 20 client.4734 trim_cache size 0 max 16384 2024-04-22T07:04:36.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.060+0000 7f101d76d740 1 Processor -- start 2024-04-22T07:04:36.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.060+0000 7f101d76d740 1 -- start start 2024-04-22T07:04:36.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f101d76d740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef4fec0 0x55e4cef95910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f101d76d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef74490 0x55e4cef95e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f101d76d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef8ee40 0x55e4cef4d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f101d76d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f10140cb2c0 con 0x55e4cef74490 2024-04-22T07:04:36.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f101d76d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e4cef965a0 con 0x55e4cef8ee40 2024-04-22T07:04:36.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f101d76d740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55e4cef96bb0 con 0x55e4cef4fec0 2024-04-22T07:04:36.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f1013fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef74490 0x55e4cef95e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f1013fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef74490 0x55e4cef95e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39230/0 (socket says 172.21.15.92:39230) 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f1013fff640 1 -- 172.21.15.92:0/3133720973 learned_addr learned my addr 172.21.15.92:0/3133720973 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f101891a640 1 --2- 172.21.15.92:0/3133720973 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef4fec0 0x55e4cef95910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f101911b640 1 --2- 172.21.15.92:0/3133720973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef8ee40 0x55e4cef4d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f1013fff640 1 -- 172.21.15.92:0/3133720973 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef4fec0 msgr2=0x55e4cef95910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f1013fff640 1 --2- 172.21.15.92:0/3133720973 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef4fec0 0x55e4cef95910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f1013fff640 1 -- 172.21.15.92:0/3133720973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef8ee40 msgr2=0x55e4cef4d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f1013fff640 1 --2- 172.21.15.92:0/3133720973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef8ee40 0x55e4cef4d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f1013fff640 1 -- 172.21.15.92:0/3133720973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4cef890e0 con 0x55e4cef74490 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f101891a640 1 --2- 172.21.15.92:0/3133720973 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef4fec0 0x55e4cef95910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.061+0000 7f101911b640 1 --2- 172.21.15.92:0/3133720973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef8ee40 0x55e4cef4d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.062+0000 7f1013fff640 1 --2- 172.21.15.92:0/3133720973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef74490 0x55e4cef95e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f100000b8d0 tx=0x7f1000030070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f2ab9427f3b076 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.062+0000 7f0ffdffb640 1 -- 172.21.15.92:0/3133720973 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000054070 con 0x55e4cef74490 2024-04-22T07:04:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.062+0000 7f0ffdffb640 1 -- 172.21.15.92:0/3133720973 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100004e2d0 con 0x55e4cef74490 2024-04-22T07:04:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.062+0000 7f0ffdffb640 1 -- 172.21.15.92:0/3133720973 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100003b040 con 0x55e4cef74490 2024-04-22T07:04:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.062+0000 7f101d76d740 1 -- 172.21.15.92:0/3133720973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef74490 msgr2=0x55e4cef95e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.062+0000 7f101d76d740 1 --2- 172.21.15.92:0/3133720973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef74490 0x55e4cef95e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f100000b8d0 tx=0x7f1000030070 comp rx=0 tx=0).stop 2024-04-22T07:04:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.062+0000 7f101d76d740 1 -- 172.21.15.92:0/3133720973 shutdown_connections 2024-04-22T07:04:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.062+0000 7f101d76d740 1 --2- 172.21.15.92:0/3133720973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef8ee40 0x55e4cef4d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.062+0000 7f101d76d740 1 --2- 172.21.15.92:0/3133720973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef74490 0x55e4cef95e50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.062+0000 7f101d76d740 1 --2- 172.21.15.92:0/3133720973 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef4fec0 0x55e4cef95910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.063+0000 7f101d76d740 1 -- 172.21.15.92:0/3133720973 >> 172.21.15.92:0/3133720973 conn(0x55e4cef4bca0 msgr2=0x55e4cefa2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.063+0000 7f101d76d740 1 -- 172.21.15.92:0/3133720973 shutdown_connections 2024-04-22T07:04:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.063+0000 7f101d76d740 1 -- 172.21.15.92:0/3133720973 wait complete. 2024-04-22T07:04:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.063+0000 7f101d76d740 1 -- start start 2024-04-22T07:04:36.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.063+0000 7f101d76d740 1 Processor -- start 2024-04-22T07:04:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101d76d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef4fec0 0x55e4cef68a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101d76d740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef74490 0x55e4cef85200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101d76d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef8ee40 0x55e4cef4d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101d76d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e4cefa3a70 con 0x55e4cef8ee40 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101d76d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e4cef6d760 con 0x55e4cef4fec0 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101d76d740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55e4cef93e30 con 0x55e4cef74490 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101d76d740 10 client.0 _ll_register_callbacks cb 0x7f1000039070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101911b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef8ee40 0x55e4cef4d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101911b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef8ee40 0x55e4cef4d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39238/0 (socket says 172.21.15.92:39238) 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101911b640 1 -- 172.21.15.92:0/298324546 learned_addr learned my addr 172.21.15.92:0/298324546 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101891a640 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef4fec0 0x55e4cef68a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f1013fff640 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef74490 0x55e4cef85200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101911b640 1 -- 172.21.15.92:0/298324546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef74490 msgr2=0x55e4cef85200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101911b640 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef74490 0x55e4cef85200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101911b640 1 -- 172.21.15.92:0/298324546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef4fec0 msgr2=0x55e4cef68a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101911b640 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef4fec0 0x55e4cef68a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.064+0000 7f101911b640 1 -- 172.21.15.92:0/298324546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4cef95530 con 0x55e4cef8ee40 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f101911b640 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef8ee40 0x55e4cef4d0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f100c0049d0 tx=0x7f100c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacd9662736550e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f0ff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f0ff37fe640 1 -- 172.21.15.92:0/298324546 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c025020 con 0x55e4cef8ee40 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f0ff37fe640 1 -- 172.21.15.92:0/298324546 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f100c00d910 con 0x55e4cef8ee40 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f101d76d740 1 -- 172.21.15.92:0/298324546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f100004e2d0 con 0x55e4cef8ee40 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f0ff37fe640 1 -- 172.21.15.92:0/298324546 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f100c02f050 con 0x55e4cef8ee40 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f101d76d740 10 client.4740 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f101d76d740 1 -- 172.21.15.92:0/298324546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55e4cef6e540 con 0x55e4cef8ee40 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f0ff37fe640 1 -- 172.21.15.92:0/298324546 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100004e2d0 con 0x55e4cef8ee40 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f0ff37fe640 1 -- 172.21.15.92:0/298324546 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f100c032020 con 0x55e4cef8ee40 2024-04-22T07:04:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f101d76d740 10 client.4740 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f101d76d740 10 client.4740 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f101d76d740 20 client.4740 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f101d76d740 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55e4cef72820 0x55e4cf017680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f101d76d740 4 client.4740 mds_command: new command op to 4229 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f101d76d740 1 -- 172.21.15.92:0/298324546 --> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] -- command(tid 0: {"prefix": "status"}) -- 0x55e4ceeba3a0 con 0x55e4cef72820 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.065+0000 7f101891a640 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55e4cef72820 0x55e4cf017680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f0ff37fe640 10 client.4740 ms_handle_connect on v2:172.21.15.145:6833/1058270486 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101891a640 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55e4cef72820 0x55e4cf017680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ceaad478a81e78cf server_cookie=147665a3003c20c2 in_seq=0 out_seq=0 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f0ff37fe640 1 -- 172.21.15.92:0/298324546 <== mds.4 v2:172.21.15.145:6833/1058270486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e4ceeba3a0 con 0x55e4cef72820 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f0ff37fe640 10 client.4740 handle_command_reply: tid=0 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101d76d740 1 client.4740 shutdown 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101d76d740 1 -- 172.21.15.92:0/298324546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef8ee40 msgr2=0x55e4cef4d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101d76d740 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef8ee40 0x55e4cef4d0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f100c0049d0 tx=0x7f100c004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101d76d740 1 -- 172.21.15.92:0/298324546 shutdown_connections 2024-04-22T07:04:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101d76d740 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e4cef8ee40 0x55e4cef4d0d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101d76d740 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55e4cef74490 0x55e4cef85200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101d76d740 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55e4cef72820 0x55e4cf017680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101d76d740 1 --2- 172.21.15.92:0/298324546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4cef4fec0 0x55e4cef68a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f0ff37fe640 0 client.4740 ms_handle_reset on v2:172.21.15.145:6833/1058270486 2024-04-22T07:04:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101d76d740 1 -- 172.21.15.92:0/298324546 >> 172.21.15.92:0/298324546 conn(0x55e4cef4bca0 msgr2=0x55e4cef6c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101d76d740 1 -- 172.21.15.92:0/298324546 shutdown_connections 2024-04-22T07:04:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101d76d740 1 -- 172.21.15.92:0/298324546 wait complete. 2024-04-22T07:04:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.066+0000 7f101d76d740 20 client.4740 trim_cache size 0 max 16384 2024-04-22T07:04:36.071 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:04:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "72eb29c5-e441-491c-9309-4fb4ef16ce27", 2024-04-22T07:04:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 4, 2024-04-22T07:04:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4229, 2024-04-22T07:04:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-04-22T07:04:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-04-22T07:04:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-04-22T07:04:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 11.042555631000001, 2024-04-22T07:04:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 20, 2024-04-22T07:04:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 20, 2024-04-22T07:04:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 39.147980365000002 2024-04-22T07:04:36.073 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:04:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.067+0000 7f101ab7e640 1 -- 172.21.15.92:0/164652441 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0ff806c920 msgr2=0x7f0ff806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.067+0000 7f101ab7e640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0ff806c920 0x7f0ff806ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1000038000 tx=0x7f1000032000 comp rx=0 tx=0).stop 2024-04-22T07:04:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.067+0000 7f101ab7e640 1 -- 172.21.15.92:0/164652441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10140c69d0 msgr2=0x7f10141ef940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.067+0000 7f101ab7e640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10140c69d0 0x7f10141ef940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f100800c7f0 tx=0x7f100800ccb0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.067+0000 7f101ab7e640 1 -- 172.21.15.92:0/164652441 shutdown_connections 2024-04-22T07:04:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.067+0000 7f101ab7e640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0ff806c920 0x7f0ff806ede0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.067+0000 7f101ab7e640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1014151d10 0x7f101414f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.067+0000 7f101ab7e640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10140c8dc0 0x7f10141ecd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.067+0000 7f101ab7e640 1 --2- 172.21.15.92:0/164652441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10140c69d0 0x7f10141ef940 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.067+0000 7f101ab7e640 1 -- 172.21.15.92:0/164652441 >> 172.21.15.92:0/164652441 conn(0x7f10140c9dd0 msgr2=0x7f101414e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.067+0000 7f101ab7e640 1 -- 172.21.15.92:0/164652441 shutdown_connections 2024-04-22T07:04:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.067+0000 7f101ab7e640 1 -- 172.21.15.92:0/164652441 wait complete. 2024-04-22T07:04:36.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-22T07:04:36.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.177+0000 7f5eca2a5640 1 Processor -- start 2024-04-22T07:04:36.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.177+0000 7f5eca2a5640 1 -- start start 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.178+0000 7f5eca2a5640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ec4067920 0x7f5ec415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.178+0000 7f5eca2a5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ec40c7e40 con 0x7f5ec40c76d0 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.178+0000 7f5eca2a5640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ec40c8100 con 0x7f5ec4067d00 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.178+0000 7f5eca2a5640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec406d370 con 0x7f5ec4067920 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.178+0000 7f5ec37fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5ec40c76d0 0x7f5ec4154b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53058/0 (socket says 172.21.15.92:53058) 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.178+0000 7f5ec37fe640 1 -- 172.21.15.92:0/547907128 learned_addr learned my addr 172.21.15.92:0/547907128 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.178+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 525040296 0 0) 0x7f5ec40c7e40 con 0x7f5ec40c76d0 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.178+0000 7f5ec3fff640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ec4067920 0x7f5ec415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.178+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5eac003520 con 0x7f5ec40c76d0 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.178+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 748729881 0 0) 0x7f5eac003520 con 0x7f5ec40c76d0 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.178+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ec4067920 msgr2=0x7f5ec415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.178+0000 7f5ec2ffd640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ec4067920 0x7f5ec415e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 >> v1:172.21.15.145:6789/0 conn(0x7f5ec4067d00 legacy=0x7f5ec4165610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec406c890 con 0x7f5ec40c76d0 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f5eb8001670 con 0x7f5ec40c76d0 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 >> v1:172.21.15.92:6789/0 conn(0x7f5ec40c76d0 legacy=0x7f5ec4154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec2ffd640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ec4067920 0x7f5eac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec2ffd640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eac003f60 0x7f5eac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec2ffd640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5eac004d40 0x7f5eac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec406d370 con 0x7f5eac003f60 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac00bb30 con 0x7f5eac004d40 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5eac00bd30 con 0x7f5ec4067920 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eac003f60 0x7f5eac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec3fff640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ec4067920 0x7f5eac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec8842640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5eac004d40 0x7f5eac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec37fe640 1 -- 172.21.15.92:0/547907128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ec4067920 msgr2=0x7f5eac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ec4067920 0x7f5eac0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec37fe640 1 -- 172.21.15.92:0/547907128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5eac004d40 msgr2=0x7f5eac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5eac004d40 0x7f5eac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec37fe640 1 -- 172.21.15.92:0/547907128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac003520 con 0x7f5eac003f60 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.179+0000 7f5ec8842640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5eac004d40 0x7f5eac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.180+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eac003f60 0x7f5eac004800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c210 tx=0x7f5eac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884ead4229693270 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.180+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8007cf0 con 0x7f5eac003f60 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.180+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb8030070 con 0x7f5eac003f60 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.180+0000 7f5ec2ffd640 1 -- 172.21.15.92:0/547907128 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8005d30 con 0x7f5eac003f60 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.180+0000 7f5eca2a5640 1 -- 172.21.15.92:0/547907128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eac003f60 msgr2=0x7f5eac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.180+0000 7f5eca2a5640 1 --2- 172.21.15.92:0/547907128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eac003f60 0x7f5eac004800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c210 tx=0x7f5eac005830 comp rx=0 tx=0).stop 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.180+0000 7f5ec8842640 1 -- 172.21.15.92:0/547907128 reap_dead start 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.180+0000 7f5eca2a5640 1 -- 172.21.15.92:0/547907128 shutdown_connections 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.180+0000 7f5eca2a5640 1 -- 172.21.15.92:0/547907128 >> 172.21.15.92:0/547907128 conn(0x7f5ec40c9f40 msgr2=0x7f5ec40ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.180+0000 7f5eca2a5640 1 -- 172.21.15.92:0/547907128 shutdown_connections 2024-04-22T07:04:36.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.180+0000 7f5eca2a5640 1 -- 172.21.15.92:0/547907128 wait complete. 2024-04-22T07:04:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.181+0000 7f5eca2a5640 1 Processor -- start 2024-04-22T07:04:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.182+0000 7f5eca2a5640 1 -- start start 2024-04-22T07:04:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.182+0000 7f5eca2a5640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5eac003f60 0x7f5ec4151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.182+0000 7f5eca2a5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ec4157d20 0x7f5ec415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.182+0000 7f5eca2a5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ec41f1990 0x7f5ec41f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.182+0000 7f5eca2a5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec406b970 con 0x7f5ec41f1990 2024-04-22T07:04:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.182+0000 7f5eca2a5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec406ce50 con 0x7f5ec4157d20 2024-04-22T07:04:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.182+0000 7f5eca2a5640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec406baf0 con 0x7f5eac003f60 2024-04-22T07:04:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.182+0000 7f5ec8842640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ec41f1990 0x7f5ec41f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.182+0000 7f5ec8842640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ec41f1990 0x7f5ec41f1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39260/0 (socket says 172.21.15.92:39260) 2024-04-22T07:04:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.182+0000 7f5ec8842640 1 -- 172.21.15.92:0/3165787468 learned_addr learned my addr 172.21.15.92:0/3165787468 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5ec3fff640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5eac003f60 0x7f5ec4151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ec4157d20 0x7f5ec415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5ec8842640 1 -- 172.21.15.92:0/3165787468 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5eac003f60 msgr2=0x7f5ec4151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5ec8842640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5eac003f60 0x7f5ec4151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5ec8842640 1 -- 172.21.15.92:0/3165787468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ec4157d20 msgr2=0x7f5ec415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5ec8842640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ec4157d20 0x7f5ec415b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5ec8842640 1 -- 172.21.15.92:0/3165787468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec41f2350 con 0x7f5ec41f1990 2024-04-22T07:04:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ec4157d20 0x7f5ec415b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5ec8842640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ec41f1990 0x7f5ec41f1e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc000f80 tx=0x7f5ebc00ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5ec0ff9640 1 -- 172.21.15.92:0/3165787468 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc003b80 con 0x7f5ec41f1990 2024-04-22T07:04:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5ec0ff9640 1 -- 172.21.15.92:0/3165787468 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ebc003d40 con 0x7f5ec41f1990 2024-04-22T07:04:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5ec0ff9640 1 -- 172.21.15.92:0/3165787468 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc006b50 con 0x7f5ec41f1990 2024-04-22T07:04:36.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5eca2a5640 1 -- 172.21.15.92:0/3165787468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec4154360 con 0x7f5ec41f1990 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.183+0000 7f5eca2a5640 1 -- 172.21.15.92:0/3165787468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec4153320 con 0x7f5ec41f1990 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.184+0000 7f5ec0ff9640 1 -- 172.21.15.92:0/3165787468 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ebc01a020 con 0x7f5ec41f1990 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.185+0000 7f5ec0ff9640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5eb006c920 0x7f5eb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.185+0000 7f5ec0ff9640 1 -- 172.21.15.92:0/3165787468 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5ebc08b140 con 0x7f5ec41f1990 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.185+0000 7f5ec3fff640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5eb006c920 0x7f5eb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.185+0000 7f5ec3fff640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5eb006c920 0x7f5eb006ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009bc0 tx=0x7f5eb4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.186+0000 7f5eccf73740 1 Processor -- start 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.186+0000 7f5eccf73740 1 -- start start 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.186+0000 7f5eccf73740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d9aae40 0x55980d9b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.186+0000 7f5eccf73740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d96bec0 0x55980d9b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.186+0000 7f5eccf73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d967ca0 0x55980d98fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.186+0000 7f5eccf73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55980d8ceb10 con 0x55980d967ca0 2024-04-22T07:04:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.186+0000 7f5eccf73740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55980d86c240 con 0x55980d9aae40 2024-04-22T07:04:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.186+0000 7f5eccf73740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55980d8caad0 con 0x55980d96bec0 2024-04-22T07:04:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.187+0000 7f5ec8842640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d967ca0 0x55980d98fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.187+0000 7f5ec8842640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d967ca0 0x55980d98fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39274/0 (socket says 172.21.15.92:39274) 2024-04-22T07:04:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.187+0000 7f5ec8842640 1 -- 172.21.15.92:0/2254015117 learned_addr learned my addr 172.21.15.92:0/2254015117 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.187+0000 7f5ec8842640 1 -- 172.21.15.92:0/2254015117 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d96bec0 msgr2=0x55980d9b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.187+0000 7f5ec8842640 1 --2- 172.21.15.92:0/2254015117 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d96bec0 0x55980d9b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.187+0000 7f5ec8842640 1 -- 172.21.15.92:0/2254015117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d9aae40 msgr2=0x55980d9b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.187+0000 7f5ec8842640 1 --2- 172.21.15.92:0/2254015117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d9aae40 0x55980d9b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.187+0000 7f5ec8842640 1 -- 172.21.15.92:0/2254015117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980d5e12d0 con 0x55980d967ca0 2024-04-22T07:04:36.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.187+0000 7f5ec8842640 1 --2- 172.21.15.92:0/2254015117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d967ca0 0x55980d98fee0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc089690 tx=0x7f5ebc058900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1369134be13fcd88 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.187+0000 7f5ea9ffb640 1 -- 172.21.15.92:0/2254015117 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc061a50 con 0x55980d967ca0 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.187+0000 7f5ea9ffb640 1 -- 172.21.15.92:0/2254015117 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ebc057170 con 0x55980d967ca0 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.187+0000 7f5ea9ffb640 1 -- 172.21.15.92:0/2254015117 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc01ab90 con 0x55980d967ca0 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.188+0000 7f5eccf73740 1 -- 172.21.15.92:0/2254015117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d967ca0 msgr2=0x55980d98fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.188+0000 7f5eccf73740 1 --2- 172.21.15.92:0/2254015117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d967ca0 0x55980d98fee0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc089690 tx=0x7f5ebc058900 comp rx=0 tx=0).stop 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.188+0000 7f5eccf73740 1 -- 172.21.15.92:0/2254015117 shutdown_connections 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.188+0000 7f5eccf73740 1 --2- 172.21.15.92:0/2254015117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d967ca0 0x55980d98fee0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.188+0000 7f5eccf73740 1 --2- 172.21.15.92:0/2254015117 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d96bec0 0x55980d9b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.188+0000 7f5eccf73740 1 --2- 172.21.15.92:0/2254015117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d9aae40 0x55980d9b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.188+0000 7f5eccf73740 1 -- 172.21.15.92:0/2254015117 >> 172.21.15.92:0/2254015117 conn(0x55980d966990 msgr2=0x55980d969290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.188+0000 7f5eccf73740 1 -- 172.21.15.92:0/2254015117 shutdown_connections 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.188+0000 7f5eccf73740 1 -- 172.21.15.92:0/2254015117 wait complete. 2024-04-22T07:04:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.188+0000 7f5eccf73740 1 -- start start 2024-04-22T07:04:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.188+0000 7f5eccf73740 1 Processor -- start 2024-04-22T07:04:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5eccf73740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 0x55980d963310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5eccf73740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9aae40 0x55980d9b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5eccf73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9b1910 0x55980d9b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5eccf73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55980d96a250 con 0x55980d9b1910 2024-04-22T07:04:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5eccf73740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55980d857190 con 0x55980d96bec0 2024-04-22T07:04:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5eccf73740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55980d96a530 con 0x55980d9aae40 2024-04-22T07:04:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5eccf73740 10 client.0 _ll_register_callbacks cb 0x55980d8528b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5ec8842640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9b1910 0x55980d9b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5ec8842640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9b1910 0x55980d9b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39288/0 (socket says 172.21.15.92:39288) 2024-04-22T07:04:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5ec8842640 1 -- 172.21.15.92:0/3058066681 learned_addr learned my addr 172.21.15.92:0/3058066681 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9aae40 0x55980d9b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5ec3fff640 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 0x55980d963310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5ec8842640 1 -- 172.21.15.92:0/3058066681 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9aae40 msgr2=0x55980d9b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5ec8842640 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9aae40 0x55980d9b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5ec8842640 1 -- 172.21.15.92:0/3058066681 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 msgr2=0x55980d963310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5ec8842640 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 0x55980d963310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.189+0000 7f5ec8842640 1 -- 172.21.15.92:0/3058066681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980d6ab6b0 con 0x55980d9b1910 2024-04-22T07:04:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5ec8842640 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9b1910 0x55980d9b0e80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc007e20 tx=0x7f5ebc00efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a913f154144fc17 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5ea2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5ea2ffd640 1 -- 172.21.15.92:0/3058066681 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc089030 con 0x55980d9b1910 2024-04-22T07:04:36.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5ea2ffd640 1 -- 172.21.15.92:0/3058066681 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5ebc072d00 con 0x55980d9b1910 2024-04-22T07:04:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5eccf73740 1 -- 172.21.15.92:0/3058066681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ebc057170 con 0x55980d9b1910 2024-04-22T07:04:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5ea2ffd640 1 -- 172.21.15.92:0/3058066681 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5ebc078300 con 0x55980d9b1910 2024-04-22T07:04:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5eccf73740 10 client.4752 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5eccf73740 1 -- 172.21.15.92:0/3058066681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55980d964830 con 0x55980d9b1910 2024-04-22T07:04:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5ea2ffd640 1 -- 172.21.15.92:0/3058066681 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ebc057170 con 0x55980d9b1910 2024-04-22T07:04:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5ea2ffd640 1 -- 172.21.15.92:0/3058066681 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f5ebc01ab90 con 0x55980d9b1910 2024-04-22T07:04:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5eccf73740 10 client.4752 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5eccf73740 10 client.4752 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T07:04:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5eccf73740 20 client.4752 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5eccf73740 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x55980d9c8360 0x55980d9ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5eccf73740 4 client.4752 mds_command: new command op to 4232 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:36.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5eccf73740 1 -- 172.21.15.92:0/3058066681 --> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55980d8dc530 con 0x55980d9c8360 2024-04-22T07:04:36.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.190+0000 7f5ec3fff640 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x55980d9c8360 0x55980d9ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.191+0000 7f5ec3fff640 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x55980d9c8360 0x55980d9ca750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d44b640471fca10d server_cookie=3f719c6679e06d81 in_seq=0 out_seq=0 2024-04-22T07:04:36.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.191+0000 7f5ea2ffd640 10 client.4752 ms_handle_connect on v2:172.21.15.145:6834/3258302920 2024-04-22T07:04:36.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.191+0000 7f5ea2ffd640 1 -- 172.21.15.92:0/3058066681 <== mds.3 v2:172.21.15.145:6834/3258302920 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55980d8dc530 con 0x55980d9c8360 2024-04-22T07:04:36.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.191+0000 7f5ea2ffd640 10 client.4752 handle_command_reply: tid=0 2024-04-22T07:04:36.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5eccf73740 1 client.4752 shutdown 2024-04-22T07:04:36.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5eccf73740 1 -- 172.21.15.92:0/3058066681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9b1910 msgr2=0x55980d9b0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5eccf73740 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9b1910 0x55980d9b0e80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc007e20 tx=0x7f5ebc00efd0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5eccf73740 1 -- 172.21.15.92:0/3058066681 shutdown_connections 2024-04-22T07:04:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5eccf73740 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x55980d9c8360 0x55980d9ca750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5eccf73740 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9b1910 0x55980d9b0e80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5eccf73740 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9aae40 0x55980d9b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5ea2ffd640 0 client.4752 ms_handle_reset on v2:172.21.15.145:6834/3258302920 2024-04-22T07:04:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5eccf73740 1 --2- 172.21.15.92:0/3058066681 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 0x55980d963310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5eccf73740 1 -- 172.21.15.92:0/3058066681 >> 172.21.15.92:0/3058066681 conn(0x55980d967ca0 msgr2=0x55980d988060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5eccf73740 1 -- 172.21.15.92:0/3058066681 shutdown_connections 2024-04-22T07:04:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5eccf73740 1 -- 172.21.15.92:0/3058066681 wait complete. 2024-04-22T07:04:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.192+0000 7f5eccf73740 20 client.4752 trim_cache size 0 max 16384 2024-04-22T07:04:36.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.210+0000 7f5eccf73740 1 Processor -- start 2024-04-22T07:04:36.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.210+0000 7f5eccf73740 1 -- start start 2024-04-22T07:04:36.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.210+0000 7f5eccf73740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 0x55980d964060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.210+0000 7f5eccf73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 0x55980d9aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.210+0000 7f5eccf73740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9b1910 0x55980d9b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.210+0000 7f5eccf73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec414a030 con 0x55980d9aae40 2024-04-22T07:04:36.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.210+0000 7f5eccf73740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55980d857190 con 0x55980d96bec0 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.210+0000 7f5eccf73740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55980d96a250 con 0x55980d9b1910 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.210+0000 7f5ec37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 0x55980d9aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.210+0000 7f5ec37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 0x55980d9aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39290/0 (socket says 172.21.15.92:39290) 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.210+0000 7f5ec37fe640 1 -- 172.21.15.92:0/981015776 learned_addr learned my addr 172.21.15.92:0/981015776 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.211+0000 7f5ec37fe640 1 -- 172.21.15.92:0/981015776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9b1910 msgr2=0x55980d9b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.211+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/981015776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9b1910 0x55980d9b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.211+0000 7f5ec37fe640 1 -- 172.21.15.92:0/981015776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 msgr2=0x55980d964060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.211+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/981015776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 0x55980d964060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.211+0000 7f5ec37fe640 1 -- 172.21.15.92:0/981015776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980d9bfab0 con 0x55980d9aae40 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.211+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/981015776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 0x55980d9aff80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c300 tx=0x7f5eb802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847e0fda4e8f7676 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.211+0000 7f5ea97fa640 1 -- 172.21.15.92:0/981015776 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8001670 con 0x55980d9aae40 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.211+0000 7f5ea97fa640 1 -- 172.21.15.92:0/981015776 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb8030070 con 0x55980d9aae40 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.211+0000 7f5ea97fa640 1 -- 172.21.15.92:0/981015776 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb802ca90 con 0x55980d9aae40 2024-04-22T07:04:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.211+0000 7f5eccf73740 1 -- 172.21.15.92:0/981015776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 msgr2=0x55980d9aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.211+0000 7f5eccf73740 1 --2- 172.21.15.92:0/981015776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 0x55980d9aff80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c300 tx=0x7f5eb802e070 comp rx=0 tx=0).stop 2024-04-22T07:04:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.212+0000 7f5eccf73740 1 -- 172.21.15.92:0/981015776 shutdown_connections 2024-04-22T07:04:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.212+0000 7f5eccf73740 1 --2- 172.21.15.92:0/981015776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9b1910 0x55980d9b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.212+0000 7f5eccf73740 1 --2- 172.21.15.92:0/981015776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 0x55980d9aff80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.212+0000 7f5eccf73740 1 --2- 172.21.15.92:0/981015776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 0x55980d964060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.212+0000 7f5eccf73740 1 -- 172.21.15.92:0/981015776 >> 172.21.15.92:0/981015776 conn(0x55980d967ca0 msgr2=0x55980d9692c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.212+0000 7f5eccf73740 1 -- 172.21.15.92:0/981015776 shutdown_connections 2024-04-22T07:04:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.212+0000 7f5eccf73740 1 -- 172.21.15.92:0/981015776 wait complete. 2024-04-22T07:04:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.212+0000 7f5eccf73740 1 -- start start 2024-04-22T07:04:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.212+0000 7f5eccf73740 1 Processor -- start 2024-04-22T07:04:36.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5eccf73740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 0x55980d989900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5eccf73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 0x55980d984880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5eccf73740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9b1910 0x55980d984dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5eccf73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55980d9a8940 con 0x55980d9aae40 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5eccf73740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55980d964e40 con 0x55980d96bec0 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5eccf73740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55980d9bf0e0 con 0x55980d9b1910 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5eccf73740 10 client.0 _ll_register_callbacks cb 0x7f5ebc0587d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5ec37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 0x55980d984880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5ec37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 0x55980d984880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39306/0 (socket says 172.21.15.92:39306) 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5ec37fe640 1 -- 172.21.15.92:0/4132651979 learned_addr learned my addr 172.21.15.92:0/4132651979 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5ec3fff640 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 0x55980d989900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5ec8842640 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9b1910 0x55980d984dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5ec37fe640 1 -- 172.21.15.92:0/4132651979 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9b1910 msgr2=0x55980d984dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9b1910 0x55980d984dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5ec37fe640 1 -- 172.21.15.92:0/4132651979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 msgr2=0x55980d989900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 0x55980d989900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5ec37fe640 1 -- 172.21.15.92:0/4132651979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980d9a94d0 con 0x55980d9aae40 2024-04-22T07:04:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5ec37fe640 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 0x55980d984880 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8001970 tx=0x7f5ec40c8590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae673453399e5913 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.213+0000 7f5ea2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5ea2ffd640 1 -- 172.21.15.92:0/4132651979 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8053060 con 0x55980d9aae40 2024-04-22T07:04:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5ea2ffd640 1 -- 172.21.15.92:0/4132651979 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5eb80049e0 con 0x55980d9aae40 2024-04-22T07:04:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5eccf73740 1 -- 172.21.15.92:0/4132651979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eb8030070 con 0x55980d9aae40 2024-04-22T07:04:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5ea2ffd640 1 -- 172.21.15.92:0/4132651979 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5eb8059050 con 0x55980d9aae40 2024-04-22T07:04:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5eccf73740 10 client.4758 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5eccf73740 1 -- 172.21.15.92:0/4132651979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55980d985300 con 0x55980d9aae40 2024-04-22T07:04:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5ea2ffd640 1 -- 172.21.15.92:0/4132651979 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb8030070 con 0x55980d9aae40 2024-04-22T07:04:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5ea2ffd640 1 -- 172.21.15.92:0/4132651979 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f5eb805c020 con 0x55980d9aae40 2024-04-22T07:04:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5eccf73740 10 client.4758 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5eccf73740 10 client.4758 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T07:04:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5eccf73740 20 client.4758 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5eccf73740 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x55980da29180 0x55980da29560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5eccf73740 4 client.4758 mds_command: new command op to 4232 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5eccf73740 1 -- 172.21.15.92:0/4132651979 --> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] -- command(tid 0: {"prefix": "status"}) -- 0x55980d8d63a0 con 0x55980da29180 2024-04-22T07:04:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.214+0000 7f5ec3fff640 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x55980da29180 0x55980da29560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5ea2ffd640 10 client.4758 ms_handle_connect on v2:172.21.15.145:6834/3258302920 2024-04-22T07:04:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5ec3fff640 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x55980da29180 0x55980da29560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ecb1ab0555f705a server_cookie=be1be8933c178555 in_seq=0 out_seq=0 2024-04-22T07:04:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5ea2ffd640 1 -- 172.21.15.92:0/4132651979 <== mds.3 v2:172.21.15.145:6834/3258302920 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55980d8d63a0 con 0x55980da29180 2024-04-22T07:04:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5ea2ffd640 10 client.4758 handle_command_reply: tid=0 2024-04-22T07:04:36.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5eccf73740 1 client.4758 shutdown 2024-04-22T07:04:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5eccf73740 1 -- 172.21.15.92:0/4132651979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 msgr2=0x55980d984880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5eccf73740 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 0x55980d984880 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8001970 tx=0x7f5ec40c8590 comp rx=0 tx=0).stop 2024-04-22T07:04:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5eccf73740 1 -- 172.21.15.92:0/4132651979 shutdown_connections 2024-04-22T07:04:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5eccf73740 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55980d9b1910 0x55980d984dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5eccf73740 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x55980da29180 0x55980da29560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5eccf73740 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55980d9aae40 0x55980d984880 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5eccf73740 1 --2- 172.21.15.92:0/4132651979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55980d96bec0 0x55980d989900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5ea2ffd640 0 client.4758 ms_handle_reset on v2:172.21.15.145:6834/3258302920 2024-04-22T07:04:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5eccf73740 1 -- 172.21.15.92:0/4132651979 >> 172.21.15.92:0/4132651979 conn(0x55980d967ca0 msgr2=0x55980d9aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5eccf73740 1 -- 172.21.15.92:0/4132651979 shutdown_connections 2024-04-22T07:04:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5eccf73740 1 -- 172.21.15.92:0/4132651979 wait complete. 2024-04-22T07:04:36.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.215+0000 7f5eccf73740 20 client.4758 trim_cache size 0 max 16384 2024-04-22T07:04:36.221 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:04:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "72eb29c5-e441-491c-9309-4fb4ef16ce27", 2024-04-22T07:04:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 3, 2024-04-22T07:04:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4232, 2024-04-22T07:04:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-04-22T07:04:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-04-22T07:04:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-04-22T07:04:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 17.223039094000001, 2024-04-22T07:04:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 20, 2024-04-22T07:04:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 20, 2024-04-22T07:04:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 39.296387256000003 2024-04-22T07:04:36.222 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:04:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.216+0000 7f5eca2a5640 1 -- 172.21.15.92:0/3165787468 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5eb006c920 msgr2=0x7f5eb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.216+0000 7f5eca2a5640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5eb006c920 0x7f5eb006ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009bc0 tx=0x7f5eb4009190 comp rx=0 tx=0).stop 2024-04-22T07:04:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.216+0000 7f5eca2a5640 1 -- 172.21.15.92:0/3165787468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ec41f1990 msgr2=0x7f5ec41f1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.216+0000 7f5eca2a5640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ec41f1990 0x7f5ec41f1e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc000f80 tx=0x7f5ebc00ead0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.216+0000 7f5eca2a5640 1 -- 172.21.15.92:0/3165787468 shutdown_connections 2024-04-22T07:04:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.216+0000 7f5eca2a5640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5eb006c920 0x7f5eb006ede0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.216+0000 7f5eca2a5640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ec41f1990 0x7f5ec41f1e10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.216+0000 7f5eca2a5640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ec4157d20 0x7f5ec415b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.216+0000 7f5eca2a5640 1 --2- 172.21.15.92:0/3165787468 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5eac003f60 0x7f5ec4151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.216+0000 7f5eca2a5640 1 -- 172.21.15.92:0/3165787468 >> 172.21.15.92:0/3165787468 conn(0x7f5ec40c9f40 msgr2=0x7f5ec4070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.216+0000 7f5eca2a5640 1 -- 172.21.15.92:0/3165787468 shutdown_connections 2024-04-22T07:04:36.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.216+0000 7f5eca2a5640 1 -- 172.21.15.92:0/3165787468 wait complete. 2024-04-22T07:04:36.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-22T07:04:36.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.324+0000 7f7e5b83f640 1 Processor -- start 2024-04-22T07:04:36.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.324+0000 7f7e5b83f640 1 -- start start 2024-04-22T07:04:36.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.325+0000 7f7e5b83f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e540c8810 0x7f7e5415b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.325+0000 7f7e5b83f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e54155310 con 0x7f7e54154f30 2024-04-22T07:04:36.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.325+0000 7f7e5b83f640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5415e670 con 0x7f7e540c73e0 2024-04-22T07:04:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.325+0000 7f7e5b83f640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f7e5415bb80 con 0x7f7e540c8810 2024-04-22T07:04:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.325+0000 7f7e58dda640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e540c8810 0x7f7e5415b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.325+0000 7f7e58dda640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e540c8810 0x7f7e5415b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39316/0 (socket says 172.21.15.92:39316) 2024-04-22T07:04:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.325+0000 7f7e58dda640 1 -- 172.21.15.92:0/2858105260 learned_addr learned my addr 172.21.15.92:0/2858105260 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.325+0000 7f7e58dda640 1 -- 172.21.15.92:0/2858105260 >> v1:172.21.15.145:6789/0 conn(0x7f7e540c73e0 legacy=0x7f7e54072ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.325+0000 7f7e58dda640 1 -- 172.21.15.92:0/2858105260 >> v1:172.21.15.92:6789/0 conn(0x7f7e54154f30 legacy=0x7f7e540c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.326+0000 7f7e58dda640 1 -- 172.21.15.92:0/2858105260 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5415e950 con 0x7f7e540c8810 2024-04-22T07:04:36.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.326+0000 7f7e4bfff640 1 -- 172.21.15.92:0/2858105260 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 300345684 0 0) 0x7f7e54155310 con 0x7f7e54154f30 2024-04-22T07:04:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.326+0000 7f7e58dda640 1 --2- 172.21.15.92:0/2858105260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e540c8810 0x7f7e5415b640 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009870 tx=0x7f7e4402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6a346f798bfc65 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.326+0000 7f7e4bfff640 1 -- 172.21.15.92:0/2858105260 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44034070 con 0x7f7e540c8810 2024-04-22T07:04:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.326+0000 7f7e4bfff640 1 -- 172.21.15.92:0/2858105260 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e44030860 con 0x7f7e540c8810 2024-04-22T07:04:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.326+0000 7f7e4bfff640 1 -- 172.21.15.92:0/2858105260 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44030c30 con 0x7f7e540c8810 2024-04-22T07:04:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.327+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2858105260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e540c8810 msgr2=0x7f7e5415b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.327+0000 7f7e5b83f640 1 --2- 172.21.15.92:0/2858105260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e540c8810 0x7f7e5415b640 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009870 tx=0x7f7e4402b9a0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.327+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2858105260 shutdown_connections 2024-04-22T07:04:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.327+0000 7f7e5b83f640 1 --2- 172.21.15.92:0/2858105260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e540c8810 0x7f7e5415b640 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.327+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2858105260 >> 172.21.15.92:0/2858105260 conn(0x7f7e540cc970 msgr2=0x7f7e540ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.327+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2858105260 shutdown_connections 2024-04-22T07:04:36.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.327+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2858105260 wait complete. 2024-04-22T07:04:36.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.328+0000 7f7e5b83f640 1 Processor -- start 2024-04-22T07:04:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.328+0000 7f7e5b83f640 1 -- start start 2024-04-22T07:04:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e5b83f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e540c73e0 0x7f7e541f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e5b83f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e540c8810 0x7f7e541e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e5b83f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e54154f30 0x7f7e5414dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e5b83f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5415ef10 con 0x7f7e540c8810 2024-04-22T07:04:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e5b83f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7e54072760 con 0x7f7e54154f30 2024-04-22T07:04:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e5b83f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7e540729b0 con 0x7f7e540c73e0 2024-04-22T07:04:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e58dda640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e540c8810 0x7f7e541e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e58dda640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e540c8810 0x7f7e541e5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39324/0 (socket says 172.21.15.92:39324) 2024-04-22T07:04:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e58dda640 1 -- 172.21.15.92:0/2777911822 learned_addr learned my addr 172.21.15.92:0/2777911822 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e595db640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e540c73e0 0x7f7e541f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e59ddc640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e54154f30 0x7f7e5414dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e58dda640 1 -- 172.21.15.92:0/2777911822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e540c73e0 msgr2=0x7f7e541f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e58dda640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e540c73e0 0x7f7e541f4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e58dda640 1 -- 172.21.15.92:0/2777911822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e54154f30 msgr2=0x7f7e5414dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e58dda640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e54154f30 0x7f7e5414dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e58dda640 1 -- 172.21.15.92:0/2777911822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e541f1db0 con 0x7f7e540c8810 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e595db640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e540c73e0 0x7f7e541f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e59ddc640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e54154f30 0x7f7e5414dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e58dda640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e540c8810 0x7f7e541e5c30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009870 tx=0x7f7e440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.329+0000 7f7e49ffb640 1 -- 172.21.15.92:0/2777911822 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44034040 con 0x7f7e540c8810 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.330+0000 7f7e49ffb640 1 -- 172.21.15.92:0/2777911822 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e44030660 con 0x7f7e540c8810 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.330+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2777911822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e541f4ac0 con 0x7f7e540c8810 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.330+0000 7f7e49ffb640 1 -- 172.21.15.92:0/2777911822 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44039cc0 con 0x7f7e540c8810 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.330+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2777911822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e541e7a70 con 0x7f7e540c8810 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.331+0000 7f7e49ffb640 1 -- 172.21.15.92:0/2777911822 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e4404b060 con 0x7f7e540c8810 2024-04-22T07:04:36.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.331+0000 7f7e49ffb640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7e3806c920 0x7f7e3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.332+0000 7f7e49ffb640 1 -- 172.21.15.92:0/2777911822 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e440adae0 con 0x7f7e540c8810 2024-04-22T07:04:36.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.332+0000 7f7e595db640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7e3806c920 0x7f7e3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.332+0000 7f7e595db640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7e3806c920 0x7f7e3806ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7e500012e0 tx=0x7f7e50009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.332+0000 7f7e5e42b740 1 Processor -- start 2024-04-22T07:04:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.332+0000 7f7e5e42b740 1 -- start start 2024-04-22T07:04:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e5e42b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffb3e40 0x55f26ffbcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e5e42b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 0x55f26ffbd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e5e42b740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ff70ca0 0x55f26ff98ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e5e42b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f26fed7b10 con 0x55f26ffb3e40 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e5e42b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f26fe75240 con 0x55f26ff74ec0 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e5e42b740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55f26fed3ad0 con 0x55f26ff70ca0 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e595db640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffb3e40 0x55f26ffbcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e595db640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffb3e40 0x55f26ffbcf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39330/0 (socket says 172.21.15.92:39330) 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e595db640 1 -- 172.21.15.92:0/2984706147 learned_addr learned my addr 172.21.15.92:0/2984706147 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e59ddc640 1 --2- 172.21.15.92:0/2984706147 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ff70ca0 0x55f26ff98ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e58dda640 1 --2- 172.21.15.92:0/2984706147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 0x55f26ffbd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e595db640 1 -- 172.21.15.92:0/2984706147 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ff70ca0 msgr2=0x55f26ff98ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e595db640 1 --2- 172.21.15.92:0/2984706147 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ff70ca0 0x55f26ff98ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e595db640 1 -- 172.21.15.92:0/2984706147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 msgr2=0x55f26ffbd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e595db640 1 --2- 172.21.15.92:0/2984706147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 0x55f26ffbd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e595db640 1 -- 172.21.15.92:0/2984706147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26fbea2d0 con 0x55f26ffb3e40 2024-04-22T07:04:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.333+0000 7f7e595db640 1 --2- 172.21.15.92:0/2984706147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffb3e40 0x55f26ffbcf60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e50001bc0 tx=0x7f7e50010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d142da2c32fafbdb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.334+0000 7f7e3effd640 1 -- 172.21.15.92:0/2984706147 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000dea0 con 0x55f26ffb3e40 2024-04-22T07:04:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.334+0000 7f7e3effd640 1 -- 172.21.15.92:0/2984706147 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e50020450 con 0x55f26ffb3e40 2024-04-22T07:04:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.334+0000 7f7e3effd640 1 -- 172.21.15.92:0/2984706147 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50020840 con 0x55f26ffb3e40 2024-04-22T07:04:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.334+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2984706147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffb3e40 msgr2=0x55f26ffbcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.334+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/2984706147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffb3e40 0x55f26ffbcf60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e50001bc0 tx=0x7f7e50010000 comp rx=0 tx=0).stop 2024-04-22T07:04:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.334+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2984706147 shutdown_connections 2024-04-22T07:04:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.334+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/2984706147 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ff70ca0 0x55f26ff98ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.334+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/2984706147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 0x55f26ffbd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.334+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/2984706147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffb3e40 0x55f26ffbcf60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.334+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2984706147 >> 172.21.15.92:0/2984706147 conn(0x55f26ff6f990 msgr2=0x55f26ff72290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.334+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2984706147 shutdown_connections 2024-04-22T07:04:36.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.334+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2984706147 wait complete. 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.335+0000 7f7e5e42b740 1 -- start start 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.335+0000 7f7e5e42b740 1 Processor -- start 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e5e42b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ff74ec0 0x55f26ffbbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e5e42b740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 0x55f26ffb8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e5e42b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ffba030 0x55f26ffad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e5e42b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f26ff73250 con 0x55f26ff74ec0 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e5e42b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f26fe60190 con 0x55f26ffba030 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e5e42b740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55f26ff73530 con 0x55f26ffb3e40 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e595db640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ff74ec0 0x55f26ffbbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e5e42b740 10 client.0 _ll_register_callbacks cb 0x55f26fe5b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e595db640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ff74ec0 0x55f26ffbbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39344/0 (socket says 172.21.15.92:39344) 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e595db640 1 -- 172.21.15.92:0/703943592 learned_addr learned my addr 172.21.15.92:0/703943592 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e59ddc640 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ffba030 0x55f26ffad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e58dda640 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 0x55f26ffb8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e595db640 1 -- 172.21.15.92:0/703943592 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 msgr2=0x55f26ffb8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e595db640 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 0x55f26ffb8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e595db640 1 -- 172.21.15.92:0/703943592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ffba030 msgr2=0x55f26ffad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e595db640 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ffba030 0x55f26ffad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e595db640 1 -- 172.21.15.92:0/703943592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f26fcb46b0 con 0x55f26ff74ec0 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e595db640 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ff74ec0 0x55f26ffbbd60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7e500208b0 tx=0x7f7e50012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce80b0410b679a19 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e33fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e33fff640 1 -- 172.21.15.92:0/703943592 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000dea0 con 0x55f26ff74ec0 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e33fff640 1 -- 172.21.15.92:0/703943592 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e5002c430 con 0x55f26ff74ec0 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.336+0000 7f7e5e42b740 1 -- 172.21.15.92:0/703943592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e50020450 con 0x55f26ff74ec0 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e33fff640 1 -- 172.21.15.92:0/703943592 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7e50034050 con 0x55f26ff74ec0 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e5e42b740 10 client.4773 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e5e42b740 1 -- 172.21.15.92:0/703943592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55f26ffb7470 con 0x55f26ff74ec0 2024-04-22T07:04:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e33fff640 1 -- 172.21.15.92:0/703943592 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e50020450 con 0x55f26ff74ec0 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e33fff640 1 -- 172.21.15.92:0/703943592 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f7e50037020 con 0x55f26ff74ec0 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e5e42b740 10 client.4773 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e5e42b740 10 client.4773 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e5e42b740 20 client.4773 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55f26ffc8030 0x55f26ffd3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e5e42b740 4 client.4773 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e5e42b740 1 -- 172.21.15.92:0/703943592 --> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f26fee5530 con 0x55f26ffc8030 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e58dda640 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55f26ffc8030 0x55f26ffd3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e33fff640 10 client.4773 ms_handle_connect on v2:172.21.15.92:6835/1444537065 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.337+0000 7f7e58dda640 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55f26ffc8030 0x55f26ffd3840 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dcc2a020e8369f79 server_cookie=c04bbab7d6733547 in_seq=0 out_seq=0 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e33fff640 1 -- 172.21.15.92:0/703943592 <== mds.2 v2:172.21.15.92:6835/1444537065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f26fee5530 con 0x55f26ffc8030 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e33fff640 10 client.4773 handle_command_reply: tid=0 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e5e42b740 1 client.4773 shutdown 2024-04-22T07:04:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e5e42b740 1 -- 172.21.15.92:0/703943592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ff74ec0 msgr2=0x55f26ffbbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ff74ec0 0x55f26ffbbd60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7e500208b0 tx=0x7f7e50012000 comp rx=0 tx=0).stop 2024-04-22T07:04:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e5e42b740 1 -- 172.21.15.92:0/703943592 shutdown_connections 2024-04-22T07:04:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55f26ffc8030 0x55f26ffd3840 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ffba030 0x55f26ffad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 0x55f26ffb8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e33fff640 0 client.4773 ms_handle_reset on v2:172.21.15.92:6835/1444537065 2024-04-22T07:04:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/703943592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ff74ec0 0x55f26ffbbd60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e5e42b740 1 -- 172.21.15.92:0/703943592 >> 172.21.15.92:0/703943592 conn(0x55f26ff70ca0 msgr2=0x55f26ff73bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e5e42b740 1 -- 172.21.15.92:0/703943592 shutdown_connections 2024-04-22T07:04:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e5e42b740 1 -- 172.21.15.92:0/703943592 wait complete. 2024-04-22T07:04:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.338+0000 7f7e5e42b740 20 client.4773 trim_cache size 0 max 16384 2024-04-22T07:04:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.356+0000 7f7e5e42b740 1 Processor -- start 2024-04-22T07:04:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.356+0000 7f7e5e42b740 1 -- start start 2024-04-22T07:04:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e5e42b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 0x55f26ffb4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e5e42b740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 0x55f26ffd8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e5e42b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 0x55f26ffd8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e5e42b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e541fcf50 con 0x55f26ffba030 2024-04-22T07:04:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e5e42b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5414e830 con 0x55f26ff74ec0 2024-04-22T07:04:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e5e42b740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55f26ffb7a30 con 0x55f26ffb3e40 2024-04-22T07:04:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e59ddc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 0x55f26ffd8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e59ddc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 0x55f26ffd8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39346/0 (socket says 172.21.15.92:39346) 2024-04-22T07:04:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e59ddc640 1 -- 172.21.15.92:0/132745650 learned_addr learned my addr 172.21.15.92:0/132745650 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e59ddc640 1 -- 172.21.15.92:0/132745650 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 msgr2=0x55f26ffd8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e59ddc640 1 --2- 172.21.15.92:0/132745650 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 0x55f26ffd8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e59ddc640 1 -- 172.21.15.92:0/132745650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 msgr2=0x55f26ffb4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e59ddc640 1 --2- 172.21.15.92:0/132745650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 0x55f26ffb4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e59ddc640 1 -- 172.21.15.92:0/132745650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26ff74cf0 con 0x55f26ffba030 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.357+0000 7f7e59ddc640 1 --2- 172.21.15.92:0/132745650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 0x55f26ffd8fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c004f10 tx=0x7f7e4c0053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e42d859546890 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.358+0000 7f7e3e7fc640 1 -- 172.21.15.92:0/132745650 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c019030 con 0x55f26ffba030 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.358+0000 7f7e3e7fc640 1 -- 172.21.15.92:0/132745650 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c012ab0 con 0x55f26ffba030 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.358+0000 7f7e3e7fc640 1 -- 172.21.15.92:0/132745650 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c012c80 con 0x55f26ffba030 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.358+0000 7f7e5e42b740 1 -- 172.21.15.92:0/132745650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 msgr2=0x55f26ffd8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.358+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/132745650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 0x55f26ffd8fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c004f10 tx=0x7f7e4c0053d0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.358+0000 7f7e5e42b740 1 -- 172.21.15.92:0/132745650 shutdown_connections 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.358+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/132745650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 0x55f26ffd8fc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.358+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/132745650 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 0x55f26ffd8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.358+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/132745650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 0x55f26ffb4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.358+0000 7f7e5e42b740 1 -- 172.21.15.92:0/132745650 >> 172.21.15.92:0/132745650 conn(0x55f26ff70ca0 msgr2=0x55f26ffad3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.358+0000 7f7e5e42b740 1 -- 172.21.15.92:0/132745650 shutdown_connections 2024-04-22T07:04:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.358+0000 7f7e5e42b740 1 -- 172.21.15.92:0/132745650 wait complete. 2024-04-22T07:04:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.359+0000 7f7e5e42b740 1 -- start start 2024-04-22T07:04:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.359+0000 7f7e5e42b740 1 Processor -- start 2024-04-22T07:04:36.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e5e42b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 0x55f270028e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e5e42b740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 0x55f27002f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e5e42b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 0x55f26ffd7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e5e42b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f26ffc7eb0 con 0x55f26ffba030 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e5e42b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f26ff6e700 con 0x55f26ff74ec0 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e5e42b740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55f26ffc8540 con 0x55f26ffb3e40 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e5e42b740 10 client.0 _ll_register_callbacks cb 0x7f7e50003c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e59ddc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 0x55f26ffd7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e59ddc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 0x55f26ffd7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39348/0 (socket says 172.21.15.92:39348) 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e59ddc640 1 -- 172.21.15.92:0/2275827159 learned_addr learned my addr 172.21.15.92:0/2275827159 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e59ddc640 1 -- 172.21.15.92:0/2275827159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 msgr2=0x55f27002f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e595db640 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 0x55f270028e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e58dda640 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 0x55f27002f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e59ddc640 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 0x55f27002f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e59ddc640 1 -- 172.21.15.92:0/2275827159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 msgr2=0x55f270028e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e59ddc640 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 0x55f270028e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e59ddc640 1 -- 172.21.15.92:0/2275827159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f27002fdc0 con 0x55f26ffba030 2024-04-22T07:04:36.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e595db640 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 0x55f270028e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e59ddc640 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 0x55f26ffd7860 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c004ee0 tx=0x7f7e4c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3376b2dc1dd6ffae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e33fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e33fff640 1 -- 172.21.15.92:0/2275827159 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c003a70 con 0x55f26ffba030 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e33fff640 1 -- 172.21.15.92:0/2275827159 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e4c003c10 con 0x55f26ffba030 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.360+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2275827159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e4c012ab0 con 0x55f26ffba030 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e33fff640 1 -- 172.21.15.92:0/2275827159 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7e4c02f050 con 0x55f26ffba030 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e5e42b740 10 client.4779 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2275827159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55f26ff722c0 con 0x55f26ffba030 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e33fff640 1 -- 172.21.15.92:0/2275827159 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c012ab0 con 0x55f26ffba030 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e33fff640 1 -- 172.21.15.92:0/2275827159 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f7e4c032020 con 0x55f26ffba030 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e5e42b740 10 client.4779 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e5e42b740 10 client.4779 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e5e42b740 20 client.4779 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55f26ff71b60 0x55f27003b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e5e42b740 4 client.4779 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2275827159 --> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] -- command(tid 0: {"prefix": "status"}) -- 0x55f26fedf3a0 con 0x55f26ff71b60 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e595db640 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55f26ff71b60 0x55f27003b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e33fff640 10 client.4779 ms_handle_connect on v2:172.21.15.92:6835/1444537065 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e595db640 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55f26ff71b60 0x55f27003b790 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f974a5d8ccc086b server_cookie=dfb42bc0dc7bdebe in_seq=0 out_seq=0 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e33fff640 1 -- 172.21.15.92:0/2275827159 <== mds.2 v2:172.21.15.92:6835/1444537065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55f26fedf3a0 con 0x55f26ff71b60 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e33fff640 10 client.4779 handle_command_reply: tid=0 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.361+0000 7f7e5e42b740 1 client.4779 shutdown 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2275827159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 msgr2=0x55f26ffd7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 0x55f26ffd7860 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c004ee0 tx=0x7f7e4c0036f0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2275827159 shutdown_connections 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55f26ff71b60 0x55f27003b790 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f26ffba030 0x55f26ffd7860 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e33fff640 0 client.4779 ms_handle_reset on v2:172.21.15.92:6835/1444537065 2024-04-22T07:04:36.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f26ffb3e40 0x55f27002f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5e42b740 1 --2- 172.21.15.92:0/2275827159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f26ff74ec0 0x55f270028e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2275827159 >> 172.21.15.92:0/2275827159 conn(0x55f26ff70ca0 msgr2=0x55f26ffb0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2275827159 shutdown_connections 2024-04-22T07:04:36.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5e42b740 1 -- 172.21.15.92:0/2275827159 wait complete. 2024-04-22T07:04:36.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5e42b740 20 client.4779 trim_cache size 0 max 16384 2024-04-22T07:04:36.366 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:04:36.366 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "72eb29c5-e441-491c-9309-4fb4ef16ce27", 2024-04-22T07:04:36.366 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2024-04-22T07:04:36.366 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4269, 2024-04-22T07:04:36.366 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-04-22T07:04:36.366 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-04-22T07:04:36.367 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-04-22T07:04:36.367 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 22.454606032000001, 2024-04-22T07:04:36.367 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:36.367 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 20, 2024-04-22T07:04:36.367 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 20, 2024-04-22T07:04:36.367 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 39.460781421 2024-04-22T07:04:36.367 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:04:36.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2777911822 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7e3806c920 msgr2=0x7f7e3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5b83f640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7e3806c920 0x7f7e3806ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7e500012e0 tx=0x7f7e50009040 comp rx=0 tx=0).stop 2024-04-22T07:04:36.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2777911822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e540c8810 msgr2=0x7f7e541e5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5b83f640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e540c8810 0x7f7e541e5c30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009870 tx=0x7f7e440046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2777911822 shutdown_connections 2024-04-22T07:04:36.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5b83f640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f7e3806c920 0x7f7e3806ede0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5b83f640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e54154f30 0x7f7e5414dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5b83f640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e540c8810 0x7f7e541e5c30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5b83f640 1 --2- 172.21.15.92:0/2777911822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e540c73e0 0x7f7e541f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2777911822 >> 172.21.15.92:0/2777911822 conn(0x7f7e540cc970 msgr2=0x7f7e540c92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2777911822 shutdown_connections 2024-04-22T07:04:36.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.362+0000 7f7e5b83f640 1 -- 172.21.15.92:0/2777911822 wait complete. 2024-04-22T07:04:36.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-22T07:04:36.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.470+0000 7fa4413ce640 1 Processor -- start 2024-04-22T07:04:36.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.470+0000 7fa4413ce640 1 -- start start 2024-04-22T07:04:36.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.471+0000 7fa4413ce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa43c15b830 0x7fa43c0d0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.471+0000 7fa4413ce640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa43c15bc70 con 0x7fa43c154fa0 2024-04-22T07:04:36.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.471+0000 7fa4413ce640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa43c15e4a0 con 0x7fa43c0d09d0 2024-04-22T07:04:36.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.471+0000 7fa4413ce640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fa43c15e780 con 0x7fa43c15b830 2024-04-22T07:04:36.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.471+0000 7fa43a7fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa43c15b830 0x7fa43c0d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.471+0000 7fa43a7fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa43c15b830 0x7fa43c0d0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39354/0 (socket says 172.21.15.92:39354) 2024-04-22T07:04:36.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.471+0000 7fa43affd640 1 --1- 172.21.15.92:0/1690055358 >> v1:172.21.15.92:6789/0 conn(0x7fa43c154fa0 0x7fa43c0cb480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53062/0 (socket says 172.21.15.92:53062) 2024-04-22T07:04:36.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.471+0000 7fa43a7fc640 1 -- 172.21.15.92:0/1690055358 learned_addr learned my addr 172.21.15.92:0/1690055358 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.471+0000 7fa43a7fc640 1 -- 172.21.15.92:0/1690055358 >> v1:172.21.15.145:6789/0 conn(0x7fa43c0d09d0 legacy=0x7fa43c0b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.471+0000 7fa43a7fc640 1 -- 172.21.15.92:0/1690055358 >> v1:172.21.15.92:6789/0 conn(0x7fa43c154fa0 legacy=0x7fa43c0cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.471+0000 7fa439ffb640 1 -- 172.21.15.92:0/1690055358 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422688514 0 0) 0x7fa43c15bc70 con 0x7fa43c154fa0 2024-04-22T07:04:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.471+0000 7fa43a7fc640 1 -- 172.21.15.92:0/1690055358 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c15ea60 con 0x7fa43c15b830 2024-04-22T07:04:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.472+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/1690055358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c15b830 0x7fa43c0d0490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa430009870 tx=0x7fa43002bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c45ba129a60439e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.472+0000 7fa439ffb640 1 -- 172.21.15.92:0/1690055358 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43002d870 con 0x7fa43c15b830 2024-04-22T07:04:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.472+0000 7fa439ffb640 1 -- 172.21.15.92:0/1690055358 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43002da10 con 0x7fa43c15b830 2024-04-22T07:04:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.472+0000 7fa439ffb640 1 -- 172.21.15.92:0/1690055358 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43002dde0 con 0x7fa43c15b830 2024-04-22T07:04:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.473+0000 7fa4413ce640 1 -- 172.21.15.92:0/1690055358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c15b830 msgr2=0x7fa43c0d0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.473+0000 7fa4413ce640 1 --2- 172.21.15.92:0/1690055358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c15b830 0x7fa43c0d0490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa430009870 tx=0x7fa43002bab0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.473+0000 7fa4413ce640 1 -- 172.21.15.92:0/1690055358 shutdown_connections 2024-04-22T07:04:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.473+0000 7fa4413ce640 1 --2- 172.21.15.92:0/1690055358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c15b830 0x7fa43c0d0490 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.473+0000 7fa4413ce640 1 -- 172.21.15.92:0/1690055358 >> 172.21.15.92:0/1690055358 conn(0x7fa43c073930 msgr2=0x7fa43c06cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.473+0000 7fa4413ce640 1 -- 172.21.15.92:0/1690055358 shutdown_connections 2024-04-22T07:04:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.473+0000 7fa4413ce640 1 -- 172.21.15.92:0/1690055358 wait complete. 2024-04-22T07:04:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.474+0000 7fa4413ce640 1 Processor -- start 2024-04-22T07:04:36.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.474+0000 7fa4413ce640 1 -- start start 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa4413ce640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa43c0d09d0 0x7fa43c1e7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa4413ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c154fa0 0x7fa43c1f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa4413ce640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa43c15b830 0x7fa43c1f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa4413ce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c0cbb90 con 0x7fa43c154fa0 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa4413ce640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c15f020 con 0x7fa43c0d09d0 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa4413ce640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa43c0cbd10 con 0x7fa43c15b830 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c154fa0 0x7fa43c1f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c154fa0 0x7fa43c1f4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39364/0 (socket says 172.21.15.92:39364) 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43a7fc640 1 -- 172.21.15.92:0/3287809538 learned_addr learned my addr 172.21.15.92:0/3287809538 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43affd640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa43c0d09d0 0x7fa43c1e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa43c15b830 0x7fa43c1f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43a7fc640 1 -- 172.21.15.92:0/3287809538 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa43c15b830 msgr2=0x7fa43c1f0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa43c15b830 0x7fa43c1f0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43a7fc640 1 -- 172.21.15.92:0/3287809538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa43c0d09d0 msgr2=0x7fa43c1e7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa43c0d09d0 0x7fa43c1e7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43a7fc640 1 -- 172.21.15.92:0/3287809538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c159130 con 0x7fa43c154fa0 2024-04-22T07:04:36.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43affd640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa43c0d09d0 0x7fa43c1e7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:36.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa43c15b830 0x7fa43c1f0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:36.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c154fa0 0x7fa43c1f4470 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa430009990 tx=0x7fa430004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa4237fe640 1 -- 172.21.15.92:0/3287809538 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430047070 con 0x7fa43c154fa0 2024-04-22T07:04:36.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa4237fe640 1 -- 172.21.15.92:0/3287809538 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa430037a40 con 0x7fa43c154fa0 2024-04-22T07:04:36.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa4413ce640 1 -- 172.21.15.92:0/3287809538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c158b90 con 0x7fa43c154fa0 2024-04-22T07:04:36.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa4237fe640 1 -- 172.21.15.92:0/3287809538 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430037e10 con 0x7fa43c154fa0 2024-04-22T07:04:36.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.475+0000 7fa4413ce640 1 -- 172.21.15.92:0/3287809538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c15dc90 con 0x7fa43c154fa0 2024-04-22T07:04:36.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.477+0000 7fa4237fe640 1 -- 172.21.15.92:0/3287809538 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa430041390 con 0x7fa43c154fa0 2024-04-22T07:04:36.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.477+0000 7fa4237fe640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa42806c920 0x7fa42806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.477+0000 7fa4237fe640 1 -- 172.21.15.92:0/3287809538 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa4300ad860 con 0x7fa43c154fa0 2024-04-22T07:04:36.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.477+0000 7fa43affd640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa42806c920 0x7fa42806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.478+0000 7fa43affd640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa42806c920 0x7fa42806ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0012e0 tx=0x7fa42c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.478+0000 7fa443fbe740 1 Processor -- start 2024-04-22T07:04:36.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.478+0000 7fa443fbe740 1 -- start start 2024-04-22T07:04:36.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa443fbe740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7b19e40 0x5630c7b22f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa443fbe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7adaec0 0x5630c7b234a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa443fbe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7ad6ca0 0x5630c7afeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa443fbe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5630c7a3db10 con 0x5630c7ad6ca0 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa443fbe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5630c79db240 con 0x5630c7adaec0 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa443fbe740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5630c7a39ad0 con 0x5630c7b19e40 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa43b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7ad6ca0 0x5630c7afeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa43b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7ad6ca0 0x5630c7afeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39374/0 (socket says 172.21.15.92:39374) 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa43b7fe640 1 -- 172.21.15.92:0/852939741 learned_addr learned my addr 172.21.15.92:0/852939741 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa43affd640 1 --2- 172.21.15.92:0/852939741 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7b19e40 0x5630c7b22f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/852939741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7adaec0 0x5630c7b234a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa43b7fe640 1 -- 172.21.15.92:0/852939741 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7b19e40 msgr2=0x5630c7b22f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/852939741 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7b19e40 0x5630c7b22f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa43b7fe640 1 -- 172.21.15.92:0/852939741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7adaec0 msgr2=0x5630c7b234a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/852939741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7adaec0 0x5630c7b234a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa43b7fe640 1 -- 172.21.15.92:0/852939741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c77502d0 con 0x5630c7ad6ca0 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.479+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/852939741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7ad6ca0 0x5630c7afeee0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa434001740 tx=0x7fa434001c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f016505ed1b1041 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.480+0000 7fa420ff9640 1 -- 172.21.15.92:0/852939741 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434018070 con 0x5630c7ad6ca0 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.480+0000 7fa420ff9640 1 -- 172.21.15.92:0/852939741 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43400d730 con 0x5630c7ad6ca0 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.480+0000 7fa420ff9640 1 -- 172.21.15.92:0/852939741 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43400dae0 con 0x5630c7ad6ca0 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.480+0000 7fa443fbe740 1 -- 172.21.15.92:0/852939741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7ad6ca0 msgr2=0x5630c7afeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.480+0000 7fa443fbe740 1 --2- 172.21.15.92:0/852939741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7ad6ca0 0x5630c7afeee0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa434001740 tx=0x7fa434001c00 comp rx=0 tx=0).stop 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.480+0000 7fa443fbe740 1 -- 172.21.15.92:0/852939741 shutdown_connections 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.480+0000 7fa443fbe740 1 --2- 172.21.15.92:0/852939741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7ad6ca0 0x5630c7afeee0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.480+0000 7fa443fbe740 1 --2- 172.21.15.92:0/852939741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7adaec0 0x5630c7b234a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.480+0000 7fa443fbe740 1 --2- 172.21.15.92:0/852939741 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7b19e40 0x5630c7b22f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.480+0000 7fa443fbe740 1 -- 172.21.15.92:0/852939741 >> 172.21.15.92:0/852939741 conn(0x5630c7ad5990 msgr2=0x5630c7ad8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.480+0000 7fa443fbe740 1 -- 172.21.15.92:0/852939741 shutdown_connections 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.480+0000 7fa443fbe740 1 -- 172.21.15.92:0/852939741 wait complete. 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.481+0000 7fa443fbe740 1 -- start start 2024-04-22T07:04:36.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.481+0000 7fa443fbe740 1 Processor -- start 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa443fbe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7adaec0 0x5630c7b1e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa443fbe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7b19e40 0x5630c7b2e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa443fbe740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7afbc80 0x5630c7b2e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa443fbe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5630c7ad9250 con 0x5630c7b19e40 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa443fbe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5630c79c6190 con 0x5630c7adaec0 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa443fbe740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5630c7ad9530 con 0x5630c7afbc80 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa443fbe740 10 client.0 _ll_register_callbacks cb 0x5630c79c18b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa43a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7b19e40 0x5630c7b2e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa43a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7b19e40 0x5630c7b2e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39376/0 (socket says 172.21.15.92:39376) 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa43a7fc640 1 -- 172.21.15.92:0/2266986545 learned_addr learned my addr 172.21.15.92:0/2266986545 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa43affd640 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7adaec0 0x5630c7b1e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7afbc80 0x5630c7b2e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa43a7fc640 1 -- 172.21.15.92:0/2266986545 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7afbc80 msgr2=0x5630c7b2e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7afbc80 0x5630c7b2e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa43a7fc640 1 -- 172.21.15.92:0/2266986545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7adaec0 msgr2=0x5630c7b1e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7adaec0 0x5630c7b1e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa43a7fc640 1 -- 172.21.15.92:0/2266986545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c781a6b0 con 0x5630c7b19e40 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7b19e40 0x5630c7b2e390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa430083c40 tx=0x7fa43007be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5556badf8cb3c13d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa419ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa419ffb640 1 -- 172.21.15.92:0/2266986545 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430083ee0 con 0x5630c7b19e40 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa419ffb640 1 -- 172.21.15.92:0/2266986545 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa43007a9e0 con 0x5630c7b19e40 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa443fbe740 1 -- 172.21.15.92:0/2266986545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa43400d730 con 0x5630c7b19e40 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa419ffb640 1 -- 172.21.15.92:0/2266986545 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa4300091c0 con 0x5630c7b19e40 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa443fbe740 10 client.4794 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.482+0000 7fa443fbe740 1 -- 172.21.15.92:0/2266986545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5630c7b21d50 con 0x5630c7b19e40 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.483+0000 7fa419ffb640 1 -- 172.21.15.92:0/2266986545 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43400d730 con 0x5630c7b19e40 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.483+0000 7fa419ffb640 1 -- 172.21.15.92:0/2266986545 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fa430041070 con 0x5630c7b19e40 2024-04-22T07:04:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.483+0000 7fa443fbe740 10 client.4794 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.483+0000 7fa443fbe740 10 client.4794 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.483+0000 7fa443fbe740 20 client.4794 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.483+0000 7fa443fbe740 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x5630c7ad3530 0x5630c7b39ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.483+0000 7fa443fbe740 4 client.4794 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.483+0000 7fa443fbe740 1 -- 172.21.15.92:0/2266986545 --> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630c7a4b530 con 0x5630c7ad3530 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.483+0000 7fa43affd640 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x5630c7ad3530 0x5630c7b39ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.483+0000 7fa419ffb640 10 client.4794 ms_handle_connect on v2:172.21.15.92:6834/101798872 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.483+0000 7fa43affd640 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x5630c7ad3530 0x5630c7b39ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2554ec113f599981 server_cookie=aa9fc534941b9ae2 in_seq=0 out_seq=0 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa419ffb640 1 -- 172.21.15.92:0/2266986545 <== mds.1 v2:172.21.15.92:6834/101798872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630c7a4b530 con 0x5630c7ad3530 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa419ffb640 10 client.4794 handle_command_reply: tid=0 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa443fbe740 1 client.4794 shutdown 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa443fbe740 1 -- 172.21.15.92:0/2266986545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7b19e40 msgr2=0x5630c7b2e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa443fbe740 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7b19e40 0x5630c7b2e390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa430083c40 tx=0x7fa43007be00 comp rx=0 tx=0).stop 2024-04-22T07:04:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa443fbe740 1 -- 172.21.15.92:0/2266986545 shutdown_connections 2024-04-22T07:04:36.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa443fbe740 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x5630c7ad3530 0x5630c7b39ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa443fbe740 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7afbc80 0x5630c7b2e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa443fbe740 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7b19e40 0x5630c7b2e390 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa443fbe740 1 --2- 172.21.15.92:0/2266986545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7adaec0 0x5630c7b1e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa419ffb640 0 client.4794 ms_handle_reset on v2:172.21.15.92:6834/101798872 2024-04-22T07:04:36.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa443fbe740 1 -- 172.21.15.92:0/2266986545 >> 172.21.15.92:0/2266986545 conn(0x5630c7ad6ca0 msgr2=0x5630c7b22ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa443fbe740 1 -- 172.21.15.92:0/2266986545 shutdown_connections 2024-04-22T07:04:36.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa443fbe740 1 -- 172.21.15.92:0/2266986545 wait complete. 2024-04-22T07:04:36.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.484+0000 7fa443fbe740 20 client.4794 trim_cache size 0 max 16384 2024-04-22T07:04:36.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.502+0000 7fa443fbe740 1 Processor -- start 2024-04-22T07:04:36.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.502+0000 7fa443fbe740 1 -- start start 2024-04-22T07:04:36.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.502+0000 7fa443fbe740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7adaec0 0x5630c7ad8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.502+0000 7fa443fbe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 0x5630c7afedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.502+0000 7fa443fbe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7b19e40 0x5630c7aff300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.502+0000 7fa443fbe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1fd150 con 0x5630c7afbc80 2024-04-22T07:04:36.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.502+0000 7fa443fbe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c150ba0 con 0x5630c7b19e40 2024-04-22T07:04:36.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.502+0000 7fa443fbe740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5630c7b22030 con 0x5630c7adaec0 2024-04-22T07:04:36.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa43a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 0x5630c7afedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa43a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 0x5630c7afedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39382/0 (socket says 172.21.15.92:39382) 2024-04-22T07:04:36.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa43a7fc640 1 -- 172.21.15.92:0/2364030713 learned_addr learned my addr 172.21.15.92:0/2364030713 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa43affd640 1 --2- 172.21.15.92:0/2364030713 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7adaec0 0x5630c7ad8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/2364030713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7b19e40 0x5630c7aff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa43a7fc640 1 -- 172.21.15.92:0/2364030713 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7adaec0 msgr2=0x5630c7ad8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/2364030713 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7adaec0 0x5630c7ad8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa43a7fc640 1 -- 172.21.15.92:0/2364030713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7b19e40 msgr2=0x5630c7aff300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/2364030713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7b19e40 0x5630c7aff300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa43a7fc640 1 -- 172.21.15.92:0/2364030713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c7ad46d0 con 0x5630c7afbc80 2024-04-22T07:04:36.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa43affd640 1 --2- 172.21.15.92:0/2364030713 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7adaec0 0x5630c7ad8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:36.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/2364030713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 0x5630c7afedc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa430006da0 tx=0x7fa430043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4de91f40d09c96e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa41bfff640 1 -- 172.21.15.92:0/2364030713 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430098070 con 0x5630c7afbc80 2024-04-22T07:04:36.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa41bfff640 1 -- 172.21.15.92:0/2364030713 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa430079630 con 0x5630c7afbc80 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.503+0000 7fa41bfff640 1 -- 172.21.15.92:0/2364030713 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430083ce0 con 0x5630c7afbc80 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.504+0000 7fa443fbe740 1 -- 172.21.15.92:0/2364030713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 msgr2=0x5630c7afedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.504+0000 7fa443fbe740 1 --2- 172.21.15.92:0/2364030713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 0x5630c7afedc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa430006da0 tx=0x7fa430043000 comp rx=0 tx=0).stop 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.504+0000 7fa443fbe740 1 -- 172.21.15.92:0/2364030713 shutdown_connections 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.504+0000 7fa443fbe740 1 --2- 172.21.15.92:0/2364030713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7b19e40 0x5630c7aff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.504+0000 7fa443fbe740 1 --2- 172.21.15.92:0/2364030713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 0x5630c7afedc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.504+0000 7fa443fbe740 1 --2- 172.21.15.92:0/2364030713 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7adaec0 0x5630c7ad8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.504+0000 7fa443fbe740 1 -- 172.21.15.92:0/2364030713 >> 172.21.15.92:0/2364030713 conn(0x5630c7ad6ca0 msgr2=0x5630c7af6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.504+0000 7fa443fbe740 1 -- 172.21.15.92:0/2364030713 shutdown_connections 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.504+0000 7fa443fbe740 1 -- 172.21.15.92:0/2364030713 wait complete. 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.504+0000 7fa443fbe740 1 -- start start 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.504+0000 7fa443fbe740 1 Processor -- start 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa443fbe740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7adaec0 0x5630c7af66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa443fbe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 0x5630c7afc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa443fbe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7b19e40 0x5630c7b94ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa443fbe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5630c7b1ebd0 con 0x5630c7afbc80 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa443fbe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5630c7afe7c0 con 0x5630c7b19e40 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa443fbe740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5630c7ad3a20 con 0x5630c7adaec0 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa443fbe740 10 client.0 _ll_register_callbacks cb 0x7fa430006cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa43a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 0x5630c7afc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa43a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 0x5630c7afc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39388/0 (socket says 172.21.15.92:39388) 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa43a7fc640 1 -- 172.21.15.92:0/566206796 learned_addr learned my addr 172.21.15.92:0/566206796 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7b19e40 0x5630c7b94ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa43affd640 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7adaec0 0x5630c7af66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa43a7fc640 1 -- 172.21.15.92:0/566206796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7adaec0 msgr2=0x5630c7af66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7adaec0 0x5630c7af66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa43a7fc640 1 -- 172.21.15.92:0/566206796 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7b19e40 msgr2=0x5630c7b94ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7b19e40 0x5630c7b94ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa43a7fc640 1 -- 172.21.15.92:0/566206796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c7af6dd0 con 0x5630c7afbc80 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa43a7fc640 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 0x5630c7afc650 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa43007b290 tx=0x7fa4300abc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e267378a4350f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.505+0000 7fa419ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa419ffb640 1 -- 172.21.15.92:0/566206796 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430098030 con 0x5630c7afbc80 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa419ffb640 1 -- 172.21.15.92:0/566206796 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa4300ab040 con 0x5630c7afbc80 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa443fbe740 1 -- 172.21.15.92:0/566206796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa430079630 con 0x5630c7afbc80 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa419ffb640 1 -- 172.21.15.92:0/566206796 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa43007aef0 con 0x5630c7afbc80 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa443fbe740 10 client.4800 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa443fbe740 1 -- 172.21.15.92:0/566206796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5630c7ace9a0 con 0x5630c7afbc80 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa419ffb640 1 -- 172.21.15.92:0/566206796 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa430079630 con 0x5630c7afbc80 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa419ffb640 1 -- 172.21.15.92:0/566206796 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fa4300772e0 con 0x5630c7afbc80 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa443fbe740 10 client.4800 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa443fbe740 10 client.4800 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-22T07:04:36.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa443fbe740 20 client.4800 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa443fbe740 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x5630c7b1cdb0 0x5630c7b1d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa443fbe740 4 client.4800 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa443fbe740 1 -- 172.21.15.92:0/566206796 --> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] -- command(tid 0: {"prefix": "status"}) -- 0x5630c7a453a0 con 0x5630c7b1cdb0 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa43affd640 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x5630c7b1cdb0 0x5630c7b1d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa419ffb640 10 client.4800 ms_handle_connect on v2:172.21.15.92:6834/101798872 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa43affd640 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x5630c7b1cdb0 0x5630c7b1d190 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9800b74bce03476e server_cookie=66404da076514306 in_seq=0 out_seq=0 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa419ffb640 1 -- 172.21.15.92:0/566206796 <== mds.1 v2:172.21.15.92:6834/101798872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5630c7a453a0 con 0x5630c7b1cdb0 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa419ffb640 10 client.4800 handle_command_reply: tid=0 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.506+0000 7fa443fbe740 1 client.4800 shutdown 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa443fbe740 1 -- 172.21.15.92:0/566206796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 msgr2=0x5630c7afc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa443fbe740 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 0x5630c7afc650 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa43007b290 tx=0x7fa4300abc60 comp rx=0 tx=0).stop 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa443fbe740 1 -- 172.21.15.92:0/566206796 shutdown_connections 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa443fbe740 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x5630c7b1cdb0 0x5630c7b1d190 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa443fbe740 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5630c7b19e40 0x5630c7b94ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa443fbe740 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630c7afbc80 0x5630c7afc650 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa419ffb640 0 client.4800 ms_handle_reset on v2:172.21.15.92:6834/101798872 2024-04-22T07:04:36.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa443fbe740 1 --2- 172.21.15.92:0/566206796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5630c7adaec0 0x5630c7af66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa443fbe740 1 -- 172.21.15.92:0/566206796 >> 172.21.15.92:0/566206796 conn(0x5630c7ad6ca0 msgr2=0x5630c7afedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa443fbe740 1 -- 172.21.15.92:0/566206796 shutdown_connections 2024-04-22T07:04:36.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa443fbe740 1 -- 172.21.15.92:0/566206796 wait complete. 2024-04-22T07:04:36.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa443fbe740 20 client.4800 trim_cache size 0 max 16384 2024-04-22T07:04:36.512 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:04:36.512 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "72eb29c5-e441-491c-9309-4fb4ef16ce27", 2024-04-22T07:04:36.512 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2024-04-22T07:04:36.513 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4273, 2024-04-22T07:04:36.513 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-04-22T07:04:36.513 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-04-22T07:04:36.513 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-04-22T07:04:36.513 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 27.695029096999999, 2024-04-22T07:04:36.513 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:36.513 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 20, 2024-04-22T07:04:36.513 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 20, 2024-04-22T07:04:36.513 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 39.605743644999997 2024-04-22T07:04:36.513 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:04:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa4413ce640 1 -- 172.21.15.92:0/3287809538 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa42806c920 msgr2=0x7fa42806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa4413ce640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa42806c920 0x7fa42806ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0012e0 tx=0x7fa42c009040 comp rx=0 tx=0).stop 2024-04-22T07:04:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa4413ce640 1 -- 172.21.15.92:0/3287809538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c154fa0 msgr2=0x7fa43c1f4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa4413ce640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c154fa0 0x7fa43c1f4470 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa430009990 tx=0x7fa430004730 comp rx=0 tx=0).stop 2024-04-22T07:04:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa4413ce640 1 -- 172.21.15.92:0/3287809538 shutdown_connections 2024-04-22T07:04:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa4413ce640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa42806c920 0x7fa42806ede0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa4413ce640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa43c15b830 0x7fa43c1f0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa4413ce640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c154fa0 0x7fa43c1f4470 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa4413ce640 1 --2- 172.21.15.92:0/3287809538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa43c0d09d0 0x7fa43c1e7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa4413ce640 1 -- 172.21.15.92:0/3287809538 >> 172.21.15.92:0/3287809538 conn(0x7fa43c073930 msgr2=0x7fa43c0625c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa4413ce640 1 -- 172.21.15.92:0/3287809538 shutdown_connections 2024-04-22T07:04:36.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.507+0000 7fa4413ce640 1 -- 172.21.15.92:0/3287809538 wait complete. 2024-04-22T07:04:36.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-22T07:04:36.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.615+0000 7fdfd4014640 1 Processor -- start 2024-04-22T07:04:36.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.615+0000 7fdfd4014640 1 -- start start 2024-04-22T07:04:36.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd4014640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfcc06d6f0 0x7fdfcc06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd4014640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfcc0b0da0 0x7fdfcc15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd4014640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfcc0ca090 con 0x7fdfcc158500 2024-04-22T07:04:36.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd4014640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfcc15e2a0 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd4014640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfcc15ee20 con 0x7fdfcc06d6f0 2024-04-22T07:04:36.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd25b1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfcc0b0da0 0x7fdfcc15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd1db0640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdfcc158500 0x7fdfcc06f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53076/0 (socket says 172.21.15.92:53076) 2024-04-22T07:04:36.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd25b1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfcc0b0da0 0x7fdfcc15ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39404/0 (socket says 172.21.15.92:39404) 2024-04-22T07:04:36.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd1db0640 1 -- 172.21.15.92:0/691311248 learned_addr learned my addr 172.21.15.92:0/691311248 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:36.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd25b1640 1 -- 172.21.15.92:0/691311248 >> v2:172.21.15.145:3300/0 conn(0x7fdfcc06d6f0 msgr2=0x7fdfcc06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd25b1640 1 --2- 172.21.15.92:0/691311248 >> v2:172.21.15.145:3300/0 conn(0x7fdfcc06d6f0 0x7fdfcc06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd25b1640 1 -- 172.21.15.92:0/691311248 >> v1:172.21.15.92:6789/0 conn(0x7fdfcc158500 legacy=0x7fdfcc06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd25b1640 1 -- 172.21.15.92:0/691311248 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc06bfb0 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.616+0000 7fdfd0dae640 1 -- 172.21.15.92:0/691311248 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2912693993 0 0) 0x7fdfcc0ca090 con 0x7fdfcc158500 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.617+0000 7fdfd25b1640 1 --2- 172.21.15.92:0/691311248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc0b0da0 0x7fdfcc15ad30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc15e420 tx=0x7fdfc402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdefa68c0983e0ba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.617+0000 7fdfd0dae640 1 -- 172.21.15.92:0/691311248 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4035020 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.617+0000 7fdfd0dae640 1 -- 172.21.15.92:0/691311248 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc4031830 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.617+0000 7fdfd0dae640 1 -- 172.21.15.92:0/691311248 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4031c00 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.618+0000 7fdfd4014640 1 -- 172.21.15.92:0/691311248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc0b0da0 msgr2=0x7fdfcc15ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.618+0000 7fdfd4014640 1 --2- 172.21.15.92:0/691311248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc0b0da0 0x7fdfcc15ad30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc15e420 tx=0x7fdfc402b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.618+0000 7fdfd4014640 1 -- 172.21.15.92:0/691311248 shutdown_connections 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.618+0000 7fdfd4014640 1 --2- 172.21.15.92:0/691311248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc0b0da0 0x7fdfcc15ad30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.618+0000 7fdfd4014640 1 --2- 172.21.15.92:0/691311248 >> v2:172.21.15.145:3300/0 conn(0x7fdfcc06d6f0 0x7fdfcc06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.618+0000 7fdfd4014640 1 -- 172.21.15.92:0/691311248 >> 172.21.15.92:0/691311248 conn(0x7fdfcc0676b0 msgr2=0x7fdfcc06db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.618+0000 7fdfd4014640 1 -- 172.21.15.92:0/691311248 shutdown_connections 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.618+0000 7fdfd4014640 1 -- 172.21.15.92:0/691311248 wait complete. 2024-04-22T07:04:36.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.619+0000 7fdfd4014640 1 Processor -- start 2024-04-22T07:04:36.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.619+0000 7fdfd4014640 1 -- start start 2024-04-22T07:04:36.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd4014640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfcc06d6f0 0x7fdfcc1f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd4014640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc0b0da0 0x7fdfcc14c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd4014640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfcc158500 0x7fdfcc0c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd4014640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc15eab0 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd4014640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc15e7d0 con 0x7fdfcc06d6f0 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd4014640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc06fcf0 con 0x7fdfcc158500 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd15af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc0b0da0 0x7fdfcc14c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd15af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc0b0da0 0x7fdfcc14c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39416/0 (socket says 172.21.15.92:39416) 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd15af640 1 -- 172.21.15.92:0/3229952655 learned_addr learned my addr 172.21.15.92:0/3229952655 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd1db0640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfcc06d6f0 0x7fdfcc1f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd15af640 1 -- 172.21.15.92:0/3229952655 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfcc158500 msgr2=0x7fdfcc0c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd25b1640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfcc158500 0x7fdfcc0c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd15af640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfcc158500 0x7fdfcc0c3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd15af640 1 -- 172.21.15.92:0/3229952655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfcc06d6f0 msgr2=0x7fdfcc1f05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd15af640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfcc06d6f0 0x7fdfcc1f05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd15af640 1 -- 172.21.15.92:0/3229952655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc14c6f0 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd15af640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc0b0da0 0x7fdfcc14c1b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00b4a0 tx=0x7fdfbc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfc27fc640 1 -- 172.21.15.92:0/3229952655 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc014070 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfc27fc640 1 -- 172.21.15.92:0/3229952655 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc010610 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfc27fc640 1 -- 172.21.15.92:0/3229952655 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc010a00 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd4014640 1 -- 172.21.15.92:0/3229952655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc15bee0 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.620+0000 7fdfd4014640 1 -- 172.21.15.92:0/3229952655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc1f0ee0 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.622+0000 7fdfc27fc640 1 -- 172.21.15.92:0/3229952655 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfbc017070 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.622+0000 7fdfc27fc640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fdfb006c920 0x7fdfb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.622+0000 7fdfc27fc640 1 -- 172.21.15.92:0/3229952655 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdfbc08b630 con 0x7fdfcc0b0da0 2024-04-22T07:04:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.622+0000 7fdfd1db0640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fdfb006c920 0x7fdfb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.623+0000 7fdfd1db0640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fdfb006c920 0x7fdfb006ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80012e0 tx=0x7fdfc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.623+0000 7fdfd6c08740 1 Processor -- start 2024-04-22T07:04:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.623+0000 7fdfd6c08740 1 -- start start 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd6c08740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a091e40 0x563e9a09af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd6c08740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a052ec0 0x563e9a09b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd6c08740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a04eca0 0x563e9a076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd6c08740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563e99fb5b10 con 0x563e9a04eca0 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd6c08740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563e99f53240 con 0x563e9a091e40 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd6c08740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x563e99fb1ad0 con 0x563e9a052ec0 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd25b1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a04eca0 0x563e9a076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd25b1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a04eca0 0x563e9a076ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39432/0 (socket says 172.21.15.92:39432) 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd25b1640 1 -- 172.21.15.92:0/1883553155 learned_addr learned my addr 172.21.15.92:0/1883553155 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd15af640 1 --2- 172.21.15.92:0/1883553155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a052ec0 0x563e9a09b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd1db0640 1 --2- 172.21.15.92:0/1883553155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a091e40 0x563e9a09af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd25b1640 1 -- 172.21.15.92:0/1883553155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a052ec0 msgr2=0x563e9a09b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd25b1640 1 --2- 172.21.15.92:0/1883553155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a052ec0 0x563e9a09b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd25b1640 1 -- 172.21.15.92:0/1883553155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a091e40 msgr2=0x563e9a09af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd25b1640 1 --2- 172.21.15.92:0/1883553155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a091e40 0x563e9a09af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd25b1640 1 -- 172.21.15.92:0/1883553155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e99cc82d0 con 0x563e9a04eca0 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd15af640 1 --2- 172.21.15.92:0/1883553155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a052ec0 0x563e9a09b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.624+0000 7fdfd25b1640 1 --2- 172.21.15.92:0/1883553155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a04eca0 0x563e9a076ee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009860 tx=0x7fdfc4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99fcd532e46845b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.625+0000 7fdfaf7fe640 1 -- 172.21.15.92:0/1883553155 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4048070 con 0x563e9a04eca0 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.625+0000 7fdfaf7fe640 1 -- 172.21.15.92:0/1883553155 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc40317e0 con 0x563e9a04eca0 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.625+0000 7fdfaf7fe640 1 -- 172.21.15.92:0/1883553155 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4031bb0 con 0x563e9a04eca0 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.625+0000 7fdfd6c08740 1 -- 172.21.15.92:0/1883553155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a04eca0 msgr2=0x563e9a076ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.625+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/1883553155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a04eca0 0x563e9a076ee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009860 tx=0x7fdfc4030000 comp rx=0 tx=0).stop 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.625+0000 7fdfd6c08740 1 -- 172.21.15.92:0/1883553155 shutdown_connections 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.625+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/1883553155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a04eca0 0x563e9a076ee0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.625+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/1883553155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a052ec0 0x563e9a09b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.625+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/1883553155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a091e40 0x563e9a09af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.625+0000 7fdfd6c08740 1 -- 172.21.15.92:0/1883553155 >> 172.21.15.92:0/1883553155 conn(0x563e9a04d990 msgr2=0x563e9a050290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.625+0000 7fdfd6c08740 1 -- 172.21.15.92:0/1883553155 shutdown_connections 2024-04-22T07:04:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.625+0000 7fdfd6c08740 1 -- 172.21.15.92:0/1883553155 wait complete. 2024-04-22T07:04:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.626+0000 7fdfd6c08740 1 -- start start 2024-04-22T07:04:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.626+0000 7fdfd6c08740 1 Processor -- start 2024-04-22T07:04:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd6c08740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a052ec0 0x563e9a04f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd6c08740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 0x563e9a08b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd6c08740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a04c550 0x563e9a08bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd6c08740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563e9a051250 con 0x563e9a091e40 2024-04-22T07:04:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd6c08740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563e99f3e190 con 0x563e9a052ec0 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd6c08740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x563e9a051530 con 0x563e9a04c550 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd6c08740 10 client.0 _ll_register_callbacks cb 0x563e99f398b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd15af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 0x563e9a08b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd15af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 0x563e9a08b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39434/0 (socket says 172.21.15.92:39434) 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd15af640 1 -- 172.21.15.92:0/3675894903 learned_addr learned my addr 172.21.15.92:0/3675894903 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd1db0640 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a052ec0 0x563e9a04f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd25b1640 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a04c550 0x563e9a08bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd15af640 1 -- 172.21.15.92:0/3675894903 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a04c550 msgr2=0x563e9a08bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd15af640 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a04c550 0x563e9a08bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd15af640 1 -- 172.21.15.92:0/3675894903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a052ec0 msgr2=0x563e9a04f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd15af640 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a052ec0 0x563e9a04f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd15af640 1 -- 172.21.15.92:0/3675894903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e99d926b0 con 0x563e9a091e40 2024-04-22T07:04:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd1db0640 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a052ec0 0x563e9a04f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd15af640 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 0x563e9a08b650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc089b80 tx=0x7fdfbc059420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d595555327d78e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfacff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfacff9640 1 -- 172.21.15.92:0/3675894903 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc01e050 con 0x563e9a091e40 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfacff9640 1 -- 172.21.15.92:0/3675894903 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdfbc010da0 con 0x563e9a091e40 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.627+0000 7fdfd6c08740 1 -- 172.21.15.92:0/3675894903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc40317e0 con 0x563e9a091e40 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfacff9640 1 -- 172.21.15.92:0/3675894903 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fdfbc05d170 con 0x563e9a091e40 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfd6c08740 10 client.4815 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfd6c08740 1 -- 172.21.15.92:0/3675894903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x563e9a0970a0 con 0x563e9a091e40 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfacff9640 1 -- 172.21.15.92:0/3675894903 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc40317e0 con 0x563e9a091e40 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfacff9640 1 -- 172.21.15.92:0/3675894903 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fdfbc058720 con 0x563e9a091e40 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfd6c08740 10 client.4815 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfd6c08740 10 client.4815 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfd6c08740 20 client.4815 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x563e9a098e40 0x563e9a0b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfd6c08740 4 client.4815 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfd6c08740 1 -- 172.21.15.92:0/3675894903 --> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e99fc3530 con 0x563e9a098e40 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfd1db0640 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x563e9a098e40 0x563e9a0b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfacff9640 10 client.4815 ms_handle_connect on v2:172.21.15.92:6838/3493022212 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.628+0000 7fdfd1db0640 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x563e9a098e40 0x563e9a0b18f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8243904d3a616e6b server_cookie=f326a1308daa3511 in_seq=0 out_seq=0 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfacff9640 1 -- 172.21.15.92:0/3675894903 <== mds.0 v2:172.21.15.92:6838/3493022212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e99fc3530 con 0x563e9a098e40 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfacff9640 10 client.4815 handle_command_reply: tid=0 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfd6c08740 1 client.4815 shutdown 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfd6c08740 1 -- 172.21.15.92:0/3675894903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 msgr2=0x563e9a08b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 0x563e9a08b650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc089b80 tx=0x7fdfbc059420 comp rx=0 tx=0).stop 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfd6c08740 1 -- 172.21.15.92:0/3675894903 shutdown_connections 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x563e9a098e40 0x563e9a0b18f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a04c550 0x563e9a08bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 0x563e9a08b650 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/3675894903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a052ec0 0x563e9a04f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfacff9640 0 client.4815 ms_handle_reset on v2:172.21.15.92:6838/3493022212 2024-04-22T07:04:36.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfd6c08740 1 -- 172.21.15.92:0/3675894903 >> 172.21.15.92:0/3675894903 conn(0x563e9a04eca0 msgr2=0x563e9a051a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfd6c08740 1 -- 172.21.15.92:0/3675894903 shutdown_connections 2024-04-22T07:04:36.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfd6c08740 1 -- 172.21.15.92:0/3675894903 wait complete. 2024-04-22T07:04:36.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.629+0000 7fdfd6c08740 20 client.4815 trim_cache size 0 max 16384 2024-04-22T07:04:36.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.647+0000 7fdfd6c08740 1 Processor -- start 2024-04-22T07:04:36.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.647+0000 7fdfd6c08740 1 -- start start 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd6c08740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a04c550 0x563e9a09ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd6c08740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a052ec0 0x563e9a09b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd6c08740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a091e40 0x563e9a090e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd6c08740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1f3d70 con 0x563e9a052ec0 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd6c08740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563e9a04ff40 con 0x563e9a04c550 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd6c08740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x563e9a09a5e0 con 0x563e9a091e40 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd15af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a052ec0 0x563e9a09b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd15af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a052ec0 0x563e9a09b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39436/0 (socket says 172.21.15.92:39436) 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd15af640 1 -- 172.21.15.92:0/2581953058 learned_addr learned my addr 172.21.15.92:0/2581953058 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd15af640 1 -- 172.21.15.92:0/2581953058 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a091e40 msgr2=0x563e9a090e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd25b1640 1 --2- 172.21.15.92:0/2581953058 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a091e40 0x563e9a090e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd1db0640 1 --2- 172.21.15.92:0/2581953058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a04c550 0x563e9a09ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd15af640 1 --2- 172.21.15.92:0/2581953058 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a091e40 0x563e9a090e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd15af640 1 -- 172.21.15.92:0/2581953058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a04c550 msgr2=0x563e9a09ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd15af640 1 --2- 172.21.15.92:0/2581953058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a04c550 0x563e9a09ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd15af640 1 -- 172.21.15.92:0/2581953058 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e9a04e720 con 0x563e9a052ec0 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.648+0000 7fdfd15af640 1 --2- 172.21.15.92:0/2581953058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a052ec0 0x563e9a09b380 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc059490 tx=0x7fdfbc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962bb776c78752fb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.649+0000 7fdfaeffd640 1 -- 172.21.15.92:0/2581953058 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc01e040 con 0x563e9a052ec0 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.649+0000 7fdfaeffd640 1 -- 172.21.15.92:0/2581953058 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc061f40 con 0x563e9a052ec0 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.649+0000 7fdfaeffd640 1 -- 172.21.15.92:0/2581953058 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc055570 con 0x563e9a052ec0 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.649+0000 7fdfd6c08740 1 -- 172.21.15.92:0/2581953058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a052ec0 msgr2=0x563e9a09b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.649+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/2581953058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a052ec0 0x563e9a09b380 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc059490 tx=0x7fdfbc01b000 comp rx=0 tx=0).stop 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.649+0000 7fdfd6c08740 1 -- 172.21.15.92:0/2581953058 shutdown_connections 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.649+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/2581953058 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a091e40 0x563e9a090e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.649+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/2581953058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a052ec0 0x563e9a09b380 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.649+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/2581953058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a04c550 0x563e9a09ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.649+0000 7fdfd6c08740 1 -- 172.21.15.92:0/2581953058 >> 172.21.15.92:0/2581953058 conn(0x563e9a04eca0 msgr2=0x563e9a04f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.649+0000 7fdfd6c08740 1 -- 172.21.15.92:0/2581953058 shutdown_connections 2024-04-22T07:04:36.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.649+0000 7fdfd6c08740 1 -- 172.21.15.92:0/2581953058 wait complete. 2024-04-22T07:04:36.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.650+0000 7fdfd6c08740 1 -- start start 2024-04-22T07:04:36.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.650+0000 7fdfd6c08740 1 Processor -- start 2024-04-22T07:04:36.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd6c08740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a04c550 0x563e9a095cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd6c08740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a052ec0 0x563e9a04ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd6c08740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 0x563e9a106d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd6c08740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563e9a098cc0 con 0x563e9a091e40 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd6c08740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563e9a0977b0 con 0x563e9a04c550 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd6c08740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x563e9a099390 con 0x563e9a052ec0 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd6c08740 10 client.0 _ll_register_callbacks cb 0x7fdfbc059120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd25b1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 0x563e9a106d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd25b1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 0x563e9a106d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39440/0 (socket says 172.21.15.92:39440) 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd25b1640 1 -- 172.21.15.92:0/815659399 learned_addr learned my addr 172.21.15.92:0/815659399 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd15af640 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a052ec0 0x563e9a04ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd1db0640 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a04c550 0x563e9a095cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd25b1640 1 -- 172.21.15.92:0/815659399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a052ec0 msgr2=0x563e9a04ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd25b1640 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a052ec0 0x563e9a04ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd25b1640 1 -- 172.21.15.92:0/815659399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a04c550 msgr2=0x563e9a095cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd25b1640 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a04c550 0x563e9a095cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd25b1640 1 -- 172.21.15.92:0/815659399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e9a1103f0 con 0x563e9a091e40 2024-04-22T07:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd25b1640 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 0x563e9a106d30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc06ec20 tx=0x7fdfc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f668527a9fab9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfacff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfacff9640 1 -- 172.21.15.92:0/815659399 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4048040 con 0x563e9a091e40 2024-04-22T07:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfacff9640 1 -- 172.21.15.92:0/815659399 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdfc40317e0 con 0x563e9a091e40 2024-04-22T07:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd6c08740 1 -- 172.21.15.92:0/815659399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfbc061f40 con 0x563e9a091e40 2024-04-22T07:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfacff9640 1 -- 172.21.15.92:0/815659399 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fdfc4031a00 con 0x563e9a091e40 2024-04-22T07:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd6c08740 10 client.4821 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.651+0000 7fdfd6c08740 1 -- 172.21.15.92:0/815659399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x563e9a095550 con 0x563e9a091e40 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfacff9640 1 -- 172.21.15.92:0/815659399 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc061f40 con 0x563e9a091e40 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfacff9640 1 -- 172.21.15.92:0/815659399 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fdfc4042070 con 0x563e9a091e40 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 10 client.4821 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 10 client.4821 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 20 client.4821 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x563e9a075f60 0x563e9a119730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 4 client.4821 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 1 -- 172.21.15.92:0/815659399 --> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] -- command(tid 0: {"prefix": "status"}) -- 0x563e99fbd3a0 con 0x563e9a075f60 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd1db0640 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x563e9a075f60 0x563e9a119730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfacff9640 10 client.4821 ms_handle_connect on v2:172.21.15.92:6838/3493022212 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd1db0640 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x563e9a075f60 0x563e9a119730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8d005c53f3da50 server_cookie=c562d13c1fa58a4d in_seq=0 out_seq=0 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfacff9640 1 -- 172.21.15.92:0/815659399 <== mds.0 v2:172.21.15.92:6838/3493022212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563e99fbd3a0 con 0x563e9a075f60 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfacff9640 10 client.4821 handle_command_reply: tid=0 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 1 client.4821 shutdown 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 1 -- 172.21.15.92:0/815659399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 msgr2=0x563e9a106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 0x563e9a106d30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc06ec20 tx=0x7fdfc4004730 comp rx=0 tx=0).stop 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 1 -- 172.21.15.92:0/815659399 shutdown_connections 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x563e9a075f60 0x563e9a119730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9a091e40 0x563e9a106d30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e9a052ec0 0x563e9a04ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfacff9640 0 client.4821 ms_handle_reset on v2:172.21.15.92:6838/3493022212 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 1 --2- 172.21.15.92:0/815659399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e9a04c550 0x563e9a095cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.652+0000 7fdfd6c08740 1 -- 172.21.15.92:0/815659399 >> 172.21.15.92:0/815659399 conn(0x563e9a04eca0 msgr2=0x563e9a090dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd6c08740 1 -- 172.21.15.92:0/815659399 shutdown_connections 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd6c08740 1 -- 172.21.15.92:0/815659399 wait complete. 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd6c08740 20 client.4821 trim_cache size 0 max 16384 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "72eb29c5-e441-491c-9309-4fb4ef16ce27", 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4275, 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 32.878805268000001, 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 20, 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 20, 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 39.747560235000002 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd4014640 1 -- 172.21.15.92:0/3229952655 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fdfb006c920 msgr2=0x7fdfb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd4014640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fdfb006c920 0x7fdfb006ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80012e0 tx=0x7fdfc8009040 comp rx=0 tx=0).stop 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd4014640 1 -- 172.21.15.92:0/3229952655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc0b0da0 msgr2=0x7fdfcc14c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd4014640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc0b0da0 0x7fdfcc14c1b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00b4a0 tx=0x7fdfbc00b960 comp rx=0 tx=0).stop 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd4014640 1 -- 172.21.15.92:0/3229952655 shutdown_connections 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd4014640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fdfb006c920 0x7fdfb006ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd4014640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfcc158500 0x7fdfcc0c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd4014640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc0b0da0 0x7fdfcc14c1b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd4014640 1 --2- 172.21.15.92:0/3229952655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfcc06d6f0 0x7fdfcc1f05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd4014640 1 -- 172.21.15.92:0/3229952655 >> 172.21.15.92:0/3229952655 conn(0x7fdfcc0676b0 msgr2=0x7fdfcc154950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd4014640 1 -- 172.21.15.92:0/3229952655 shutdown_connections 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.653+0000 7fdfd4014640 1 -- 172.21.15.92:0/3229952655 wait complete. 2024-04-22T07:04:36.664 INFO:teuthology.run_tasks:Running task sequential... 2024-04-22T07:04:36.678 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-22T07:04:36.678 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-22T07:04:36.679 INFO:teuthology.task.print:Enabling mgr modules 2024-04-22T07:04:36.679 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-22T07:04:36.704 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-22T07:04:36.704 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-22T07:04:36.705 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-22T07:04:36.705 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-22T07:04:36.705 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:36.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:36.705 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-04-22T07:04:36.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:36.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2024-04-22T07:04:36.786 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2024-04-22T07:04:36.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:04:36.787 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-22T07:04:36.787 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-22T07:04:36.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.926+0000 7ff84759e640 1 Processor -- start 2024-04-22T07:04:36.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.926+0000 7ff84759e640 1 -- start start 2024-04-22T07:04:36.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.927+0000 7ff84759e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff848154cb0 0x7ff84815b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.927+0000 7ff84759e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff84815ba90 con 0x7ff8480c52e0 2024-04-22T07:04:36.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.927+0000 7ff84759e640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8480cb840 con 0x7ff8480cb2b0 2024-04-22T07:04:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.927+0000 7ff84759e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7ff84815ed70 con 0x7ff848154cb0 2024-04-22T07:04:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.927+0000 7ff84659c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff848154cb0 0x7ff84815b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.927+0000 7ff84659c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff848154cb0 0x7ff84815b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39452/0 (socket says 172.21.15.92:39452) 2024-04-22T07:04:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.927+0000 7ff84659c640 1 -- 172.21.15.92:0/3351301758 learned_addr learned my addr 172.21.15.92:0/3351301758 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.927+0000 7ff846d9d640 1 --1- 172.21.15.92:0/3351301758 >> v1:172.21.15.92:6789/0 conn(0x7ff8480c52e0 0x7ff8480c5730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53078/0 (socket says 172.21.15.92:53078) 2024-04-22T07:04:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.927+0000 7ff84659c640 1 -- 172.21.15.92:0/3351301758 >> v1:172.21.15.145:6789/0 conn(0x7ff8480cb2b0 legacy=0x7ff8480ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.927+0000 7ff84659c640 1 -- 172.21.15.92:0/3351301758 >> v1:172.21.15.92:6789/0 conn(0x7ff8480c52e0 legacy=0x7ff8480c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.927+0000 7ff845d9b640 1 -- 172.21.15.92:0/3351301758 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 177572806 0 0) 0x7ff840001490 con 0x7ff8480c52e0 2024-04-22T07:04:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.927+0000 7ff84659c640 1 -- 172.21.15.92:0/3351301758 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84815e160 con 0x7ff848154cb0 2024-04-22T07:04:36.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.928+0000 7ff84659c640 1 --2- 172.21.15.92:0/3351301758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff848154cb0 0x7ff84815b550 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff838009870 tx=0x7ff83802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c45301cd3a6aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.928+0000 7ff845d9b640 1 -- 172.21.15.92:0/3351301758 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83802f840 con 0x7ff848154cb0 2024-04-22T07:04:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.928+0000 7ff845d9b640 1 -- 172.21.15.92:0/3351301758 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff83802f9e0 con 0x7ff848154cb0 2024-04-22T07:04:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.928+0000 7ff845d9b640 1 -- 172.21.15.92:0/3351301758 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83802fdb0 con 0x7ff848154cb0 2024-04-22T07:04:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.928+0000 7ff84759e640 1 -- 172.21.15.92:0/3351301758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff848154cb0 msgr2=0x7ff84815b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.928+0000 7ff84759e640 1 --2- 172.21.15.92:0/3351301758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff848154cb0 0x7ff84815b550 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff838009870 tx=0x7ff83802ba20 comp rx=0 tx=0).stop 2024-04-22T07:04:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.928+0000 7ff84759e640 1 -- 172.21.15.92:0/3351301758 shutdown_connections 2024-04-22T07:04:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.928+0000 7ff84759e640 1 --2- 172.21.15.92:0/3351301758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff848154cb0 0x7ff84815b550 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.929+0000 7ff84759e640 1 -- 172.21.15.92:0/3351301758 >> 172.21.15.92:0/3351301758 conn(0x7ff848073e20 msgr2=0x7ff84806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.929+0000 7ff84759e640 1 -- 172.21.15.92:0/3351301758 shutdown_connections 2024-04-22T07:04:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.929+0000 7ff84759e640 1 -- 172.21.15.92:0/3351301758 wait complete. 2024-04-22T07:04:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.930+0000 7ff84759e640 1 Processor -- start 2024-04-22T07:04:36.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.930+0000 7ff84759e640 1 -- start start 2024-04-22T07:04:36.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.930+0000 7ff84759e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8480c52e0 0x7ff848153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.930+0000 7ff84759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8480cb2b0 0x7ff84815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff84759e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff848154cb0 0x7ff8481f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff84759e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff84815ea00 con 0x7ff8480cb2b0 2024-04-22T07:04:36.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff84759e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff84815e720 con 0x7ff8480c52e0 2024-04-22T07:04:36.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff84759e640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff848155090 con 0x7ff848154cb0 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff83ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8480cb2b0 0x7ff84815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff83ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8480cb2b0 0x7ff84815bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39454/0 (socket says 172.21.15.92:39454) 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff83ffff640 1 -- 172.21.15.92:0/2871473448 learned_addr learned my addr 172.21.15.92:0/2871473448 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff846d9d640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff848154cb0 0x7ff8481f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff84659c640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8480c52e0 0x7ff848153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff83ffff640 1 -- 172.21.15.92:0/2871473448 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff848154cb0 msgr2=0x7ff8481f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff83ffff640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff848154cb0 0x7ff8481f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff83ffff640 1 -- 172.21.15.92:0/2871473448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8480c52e0 msgr2=0x7ff848153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff83ffff640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8480c52e0 0x7ff848153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff83ffff640 1 -- 172.21.15.92:0/2871473448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84815d2f0 con 0x7ff8480cb2b0 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff84659c640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8480c52e0 0x7ff848153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff83ffff640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8480cb2b0 0x7ff84815bc90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff83000c480 tx=0x7ff83000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff83effd640 1 -- 172.21.15.92:0/2871473448 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8300052d0 con 0x7ff8480cb2b0 2024-04-22T07:04:36.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff83effd640 1 -- 172.21.15.92:0/2871473448 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff830005470 con 0x7ff8480cb2b0 2024-04-22T07:04:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff83effd640 1 -- 172.21.15.92:0/2871473448 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830012870 con 0x7ff8480cb2b0 2024-04-22T07:04:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff84759e640 1 -- 172.21.15.92:0/2871473448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8481f4740 con 0x7ff8480cb2b0 2024-04-22T07:04:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.931+0000 7ff84759e640 1 -- 172.21.15.92:0/2871473448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8480c93a0 con 0x7ff8480cb2b0 2024-04-22T07:04:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.932+0000 7ff84759e640 1 -- 172.21.15.92:0/2871473448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84804fd50 con 0x7ff8480cb2b0 2024-04-22T07:04:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.933+0000 7ff83effd640 1 -- 172.21.15.92:0/2871473448 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff830012a10 con 0x7ff8480cb2b0 2024-04-22T07:04:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.933+0000 7ff83effd640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff82806c8f0 0x7ff82806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.933+0000 7ff83effd640 1 -- 172.21.15.92:0/2871473448 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff83008b750 con 0x7ff8480cb2b0 2024-04-22T07:04:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.933+0000 7ff84659c640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff82806c8f0 0x7ff82806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.934+0000 7ff84659c640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff82806c8f0 0x7ff82806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff838036000 tx=0x7ff838039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:36.937+0000 7ff83effd640 1 -- 172.21.15.92:0/2871473448 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8300595c0 con 0x7ff8480cb2b0 2024-04-22T07:04:37.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.150+0000 7ff84759e640 1 -- 172.21.15.92:0/2871473448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff8480cbe10 con 0x7ff8480cb2b0 2024-04-22T07:04:37.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.151+0000 7ff83effd640 1 -- 172.21.15.92:0/2871473448 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7ff83005d8c0 con 0x7ff8480cb2b0 2024-04-22T07:04:37.149 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-22T07:04:37.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.153+0000 7ff84759e640 1 -- 172.21.15.92:0/2871473448 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff82806c8f0 msgr2=0x7ff82806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:37.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.153+0000 7ff84759e640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff82806c8f0 0x7ff82806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff838036000 tx=0x7ff838039040 comp rx=0 tx=0).stop 2024-04-22T07:04:37.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.153+0000 7ff84759e640 1 -- 172.21.15.92:0/2871473448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8480cb2b0 msgr2=0x7ff84815bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:37.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.153+0000 7ff84759e640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8480cb2b0 0x7ff84815bc90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff83000c480 tx=0x7ff83000c940 comp rx=0 tx=0).stop 2024-04-22T07:04:37.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.153+0000 7ff84759e640 1 -- 172.21.15.92:0/2871473448 shutdown_connections 2024-04-22T07:04:37.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.153+0000 7ff84759e640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff82806c8f0 0x7ff82806edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:37.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.153+0000 7ff84759e640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff848154cb0 0x7ff8481f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:37.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.153+0000 7ff84759e640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8480cb2b0 0x7ff84815bc90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:37.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.153+0000 7ff84759e640 1 --2- 172.21.15.92:0/2871473448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8480c52e0 0x7ff848153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:37.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.153+0000 7ff84759e640 1 -- 172.21.15.92:0/2871473448 >> 172.21.15.92:0/2871473448 conn(0x7ff848073e20 msgr2=0x7ff84814ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:37.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.153+0000 7ff84759e640 1 -- 172.21.15.92:0/2871473448 shutdown_connections 2024-04-22T07:04:37.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37.153+0000 7ff84759e640 1 -- 172.21.15.92:0/2871473448 wait complete. 2024-04-22T07:04:37.161 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-22T07:04:37.162 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-22T07:04:37.162 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2024-04-22T07:04:37.162 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-22T07:04:37.162 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:37.162 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-22T07:04:37.162 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-22T07:04:37.162 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-22T07:04:37.162 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-22T07:04:37.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:37.162 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2024-04-22T07:04:37.177 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-22T07:04:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-22T07:04:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2024-04-22T07:04:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-04-22T07:04:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2024-04-22T07:04:37.177 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-04-22T07:04:37.177 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42691sec preferred_lft 42691sec 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::566d:71ba:5916:ab31/64 scope link noprefixroute 2024-04-22T07:04:37.178 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-04-22T07:04:37.179 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-22T07:04:37.179 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:04:37.179 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-04-22T07:04:37.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2024-04-22T07:04:37.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2024-04-22T07:04:37.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2024-04-22T07:04:37.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-22T07:04:37.179 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-04-22T07:04:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:04:37.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:04:37.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:37.361 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-22T07:04:37.384 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-04-22T07:04:37.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:37.385 DEBUG:teuthology.orchestra.run.smithi092:> route 2024-04-22T07:04:37.451 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2024-04-22T07:04:37.452 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-22T07:04:37.452 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-22T07:04:37.452 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-22T07:04:37.452 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-22T07:04:37.452 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:04:37.452 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-04-22T07:04:37.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-22T07:04:37.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-22T07:04:37.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-22T07:04:37.453 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-04-22T07:04:37.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:04:37.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:04:37.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:37.725 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-04-22T07:04:37.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:37.740 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2024-04-22T07:04:37.794 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:04:37.795 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-04-22T07:04:37.795 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:04:37.795 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-22T07:04:37.795 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-04-22T07:04:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:04:37.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:04:37.914 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-22T07:04:37.915 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:04:37.915 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-04-22T07:04:37.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-22T07:04:37.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-22T07:04:37.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-22T07:04:37.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-22T07:04:37.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-22T07:04:37.916 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-04-22T07:04:37.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:04:38.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:04:38.043 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:04:38.043 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-04-22T07:04:38.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2024-04-22T07:04:38.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2024-04-22T07:04:38.043 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-04-22T07:04:38.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:04:38.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:04:38.119 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-22T07:04:38.120 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T07:04:38.120 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:38.144 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-22T07:04:38.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T07:04:38.145 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:38.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-04-22T07:04:38.264 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T07:04:38.321 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-04-22T07:04:38.321 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-04-22T07:04:38.321 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-04-22T07:04:38.322 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2024-04-22T07:04:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-04-22T07:04:38.324 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:04:38.324 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:04:38.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:38.324 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-04-22T07:04:38.376 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-22T07:04:38.376 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-22T07:04:38.418 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-04-22T07:04:38.442 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T07:04:38.497 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-04-22T07:04:38.498 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-04-22T07:04:38.498 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-04-22T07:04:38.498 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-04-22T07:04:38.498 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-04-22T07:04:38.498 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-04-22T07:04:38.498 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-04-22T07:04:38.498 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-04-22T07:04:38.498 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-04-22T07:04:38.498 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-04-22T07:04:38.498 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-04-22T07:04:38.498 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-04-22T07:04:38.498 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-04-22T07:04:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2024-04-22T07:04:38.500 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2024-04-22T07:04:38.500 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2024-04-22T07:04:38.500 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2024-04-22T07:04:38.500 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-04-22T07:04:38.500 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:04:38.500 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:04:38.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:38.500 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-04-22T07:04:38.508 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:2024-04-22T07:04:38.510+0000 7fe94a1e6480 -1 init, newargv = 0x561d191586b0 newargc=15 2024-04-22T07:04:38.508 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[43326]: starting ceph client 2024-04-22T07:04:38.521 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[43326]: starting fuse 2024-04-22T07:04:38.523 INFO:teuthology.orchestra.run.smithi092.stdout:43 2024-04-22T07:04:38.524 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-22T07:04:38.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo stdin-killer -- python3 -c ' 2024-04-22T07:04:38.524 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2024-04-22T07:04:38.524 DEBUG:teuthology.orchestra.run.smithi092:> import re 2024-04-22T07:04:38.524 DEBUG:teuthology.orchestra.run.smithi092:> import os 2024-04-22T07:04:38.524 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2024-04-22T07:04:38.524 DEBUG:teuthology.orchestra.run.smithi092:> 2024-04-22T07:04:38.524 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2024-04-22T07:04:38.524 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-22T07:04:38.524 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> return f 2024-04-22T07:04:38.525 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-22T07:04:38.526 DEBUG:teuthology.orchestra.run.smithi092:> 2024-04-22T07:04:38.526 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.0")) 2024-04-22T07:04:38.526 DEBUG:teuthology.orchestra.run.smithi092:> ' 2024-04-22T07:04:38.641 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.0.43326.asok 2024-04-22T07:04:38.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:04:38.651 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43326.asok 2024-04-22T07:04:38.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:38.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43326.asok status 2024-04-22T07:04:38.785 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:04:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2024-04-22T07:04:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "86f7587a5a09af35f0895e1a2d08527638fae697", 2024-04-22T07:04:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev)", 2024-04-22T07:04:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "0", 2024-04-22T07:04:38.785 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "43326", 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/" 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4833, 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 4833 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-22T07:04:38.786 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 849741108 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 849741108 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.4833 192.168.0.1:0/849741108", 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "192.168.0.1:0/849741108", 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 1, 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 17, 2024-04-22T07:04:38.787 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 20, 2024-04-22T07:04:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2024-04-22T07:04:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2024-04-22T07:04:38.788 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2024-04-22T07:04:38.788 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:04:38.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:38.794 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2024-04-22T07:04:38.794 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:38.808 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2024-04-22T07:04:38.809 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:38.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:38.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:38.874 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-22T07:04:38.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:39.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.016+0000 7f96f9893640 1 Processor -- start 2024-04-22T07:04:39.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.016+0000 7f96f9893640 1 -- start start 2024-04-22T07:04:39.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.017+0000 7f96f9893640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96f40d0b70 0x7f96f4154a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.017+0000 7f96f9893640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f96f40d0f50 0x7f96f4154f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.017+0000 7f96f9893640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f415ab30 con 0x7f96f40cb120 2024-04-22T07:04:39.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.017+0000 7f96f9893640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f96f415ba00 con 0x7f96f40d0f50 2024-04-22T07:04:39.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.017+0000 7f96f9893640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f96f415adf0 con 0x7f96f40d0b70 2024-04-22T07:04:39.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.017+0000 7f96f37fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f96f40cb120 0x7f96f415e2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48282/0 (socket says 172.21.15.92:48282) 2024-04-22T07:04:39.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.017+0000 7f96f37fe640 1 -- 172.21.15.92:0/2078187121 learned_addr learned my addr 172.21.15.92:0/2078187121 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:39.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.017+0000 7f96f2ffd640 1 --2- 172.21.15.92:0/2078187121 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96f40d0b70 0x7f96f4154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.017+0000 7f96f27fc640 1 --2- 172.21.15.92:0/2078187121 >> v2:172.21.15.145:3300/0 conn(0x7f96f40d0f50 0x7f96f4154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.017+0000 7f96f1ffb640 1 -- 172.21.15.92:0/2078187121 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4196746361 0 0) 0x7f96f415ab30 con 0x7f96f40cb120 2024-04-22T07:04:39.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.017+0000 7f96f1ffb640 1 -- 172.21.15.92:0/2078187121 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96dc003520 con 0x7f96f40cb120 2024-04-22T07:04:39.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.018+0000 7f96f1ffb640 1 -- 172.21.15.92:0/2078187121 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2948363956 0 0) 0x7f96dc003520 con 0x7f96f40cb120 2024-04-22T07:04:39.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.018+0000 7f96f27fc640 1 -- 172.21.15.92:0/2078187121 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96f40d0b70 msgr2=0x7f96f4154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:39.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.018+0000 7f96f27fc640 1 --2- 172.21.15.92:0/2078187121 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96f40d0b70 0x7f96f4154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.018+0000 7f96f27fc640 1 -- 172.21.15.92:0/2078187121 >> v1:172.21.15.92:6789/0 conn(0x7f96f40cb120 legacy=0x7f96f415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.018+0000 7f96f27fc640 1 -- 172.21.15.92:0/2078187121 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f415b0d0 con 0x7f96f40d0f50 2024-04-22T07:04:39.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.018+0000 7f96f27fc640 1 --2- 172.21.15.92:0/2078187121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96f40d0f50 0x7f96f4154f40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96e80097a0 tx=0x7f96e802b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a68e415af204f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:39.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.019+0000 7f96f1ffb640 1 -- 172.21.15.92:0/2078187121 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8037070 con 0x7f96f40d0f50 2024-04-22T07:04:39.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.019+0000 7f96f1ffb640 1 -- 172.21.15.92:0/2078187121 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e8030510 con 0x7f96f40d0f50 2024-04-22T07:04:39.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.019+0000 7f96f1ffb640 1 -- 172.21.15.92:0/2078187121 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec001670 con 0x7f96f40d0f50 2024-04-22T07:04:39.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.019+0000 7f96f9893640 1 -- 172.21.15.92:0/2078187121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96f40d0f50 msgr2=0x7f96f4154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.019+0000 7f96f9893640 1 --2- 172.21.15.92:0/2078187121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96f40d0f50 0x7f96f4154f40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96e80097a0 tx=0x7f96e802b710 comp rx=0 tx=0).stop 2024-04-22T07:04:39.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.019+0000 7f96f9893640 1 -- 172.21.15.92:0/2078187121 shutdown_connections 2024-04-22T07:04:39.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.019+0000 7f96f9893640 1 --2- 172.21.15.92:0/2078187121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96f40d0f50 0x7f96f4154f40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.019+0000 7f96f9893640 1 --2- 172.21.15.92:0/2078187121 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96f40d0b70 0x7f96f4154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.019+0000 7f96f9893640 1 -- 172.21.15.92:0/2078187121 >> 172.21.15.92:0/2078187121 conn(0x7f96f40736b0 msgr2=0x7f96f4073ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:39.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.019+0000 7f96f9893640 1 -- 172.21.15.92:0/2078187121 shutdown_connections 2024-04-22T07:04:39.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.019+0000 7f96f9893640 1 -- 172.21.15.92:0/2078187121 wait complete. 2024-04-22T07:04:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.020+0000 7f96f9893640 1 Processor -- start 2024-04-22T07:04:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.020+0000 7f96f9893640 1 -- start start 2024-04-22T07:04:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.021+0000 7f96f9893640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96f40cb120 0x7f96f41519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.021+0000 7f96f9893640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f40d0b70 0x7f96f41f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.021+0000 7f96f9893640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96f40d0f50 0x7f96f40b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.021+0000 7f96f9893640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f96f40cae00 con 0x7f96f40d0b70 2024-04-22T07:04:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.021+0000 7f96f9893640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f96f415b690 con 0x7f96f40cb120 2024-04-22T07:04:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.021+0000 7f96f9893640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f96f40caf80 con 0x7f96f40d0f50 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.021+0000 7f96f27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f40d0b70 0x7f96f41f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.021+0000 7f96f27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f40d0b70 0x7f96f41f42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51382/0 (socket says 172.21.15.92:51382) 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.021+0000 7f96f27fc640 1 -- 172.21.15.92:0/1885399463 learned_addr learned my addr 172.21.15.92:0/1885399463 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.021+0000 7f96f37fe640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96f40d0f50 0x7f96f40b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.021+0000 7f96f2ffd640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96f40cb120 0x7f96f41519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.022+0000 7f96f27fc640 1 -- 172.21.15.92:0/1885399463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96f40d0f50 msgr2=0x7f96f40b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.022+0000 7f96f27fc640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96f40d0f50 0x7f96f40b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.022+0000 7f96f27fc640 1 -- 172.21.15.92:0/1885399463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96f40cb120 msgr2=0x7f96f41519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.022+0000 7f96f27fc640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96f40cb120 0x7f96f41519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.022+0000 7f96f27fc640 1 -- 172.21.15.92:0/1885399463 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f415d9d0 con 0x7f96f40d0b70 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.022+0000 7f96f2ffd640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96f40cb120 0x7f96f41519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.022+0000 7f96f27fc640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f40d0b70 0x7f96f41f42c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f96e8000c00 tx=0x7f96e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.022+0000 7f96dbfff640 1 -- 172.21.15.92:0/1885399463 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8041020 con 0x7f96f40d0b70 2024-04-22T07:04:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.022+0000 7f96dbfff640 1 -- 172.21.15.92:0/1885399463 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e8032700 con 0x7f96f40d0b70 2024-04-22T07:04:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.022+0000 7f96f9893640 1 -- 172.21.15.92:0/1885399463 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f415d6f0 con 0x7f96f40d0b70 2024-04-22T07:04:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.022+0000 7f96dbfff640 1 -- 172.21.15.92:0/1885399463 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8037040 con 0x7f96f40d0b70 2024-04-22T07:04:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.022+0000 7f96f9893640 1 -- 172.21.15.92:0/1885399463 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f415d310 con 0x7f96f40d0b70 2024-04-22T07:04:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.023+0000 7f96f9893640 1 -- 172.21.15.92:0/1885399463 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f4069930 con 0x7f96f40d0b70 2024-04-22T07:04:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.023+0000 7f96dbfff640 1 -- 172.21.15.92:0/1885399463 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96e8044020 con 0x7f96f40d0b70 2024-04-22T07:04:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.024+0000 7f96dbfff640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f96e006c920 0x7f96e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.024+0000 7f96dbfff640 1 -- 172.21.15.92:0/1885399463 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f96e80ad230 con 0x7f96f40d0b70 2024-04-22T07:04:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.024+0000 7f96f2ffd640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f96e006c920 0x7f96e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.024+0000 7f96f2ffd640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f96e006c920 0x7f96e006ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96e40074c0 tx=0x7f96e400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:39.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.028+0000 7f96dbfff640 1 -- 172.21.15.92:0/1885399463 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96e804b070 con 0x7f96f40d0b70 2024-04-22T07:04:39.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.205+0000 7f96f9893640 1 -- 172.21.15.92:0/1885399463 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96f40c68a0 con 0x7f96f40d0b70 2024-04-22T07:04:39.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.206+0000 7f96dbfff640 1 -- 172.21.15.92:0/1885399463 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f96e807b5d0 con 0x7f96f40d0b70 2024-04-22T07:04:39.204 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:39.204 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:04:04.784969+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.208+0000 7f96f9893640 1 -- 172.21.15.92:0/1885399463 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f96e006c920 msgr2=0x7f96e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.208+0000 7f96f9893640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f96e006c920 0x7f96e006ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96e40074c0 tx=0x7f96e400b040 comp rx=0 tx=0).stop 2024-04-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.208+0000 7f96f9893640 1 -- 172.21.15.92:0/1885399463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f40d0b70 msgr2=0x7f96f41f42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.208+0000 7f96f9893640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f40d0b70 0x7f96f41f42c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f96e8000c00 tx=0x7f96e80046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.208+0000 7f96f9893640 1 -- 172.21.15.92:0/1885399463 shutdown_connections 2024-04-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.208+0000 7f96f9893640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f96e006c920 0x7f96e006ede0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.208+0000 7f96f9893640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96f40d0f50 0x7f96f40b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.208+0000 7f96f9893640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96f40d0b70 0x7f96f41f42c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.208+0000 7f96f9893640 1 --2- 172.21.15.92:0/1885399463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96f40cb120 0x7f96f41519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.208+0000 7f96f9893640 1 -- 172.21.15.92:0/1885399463 >> 172.21.15.92:0/1885399463 conn(0x7f96f40736b0 msgr2=0x7f96f41547d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.208+0000 7f96f9893640 1 -- 172.21.15.92:0/1885399463 shutdown_connections 2024-04-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.208+0000 7f96f9893640 1 -- 172.21.15.92:0/1885399463 wait complete. 2024-04-22T07:04:39.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:03:57.947601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:01.058111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:02.085602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:39.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:39.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.319+0000 7f0cf0231640 1 Processor -- start 2024-04-22T07:04:39.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.319+0000 7f0cf0231640 1 -- start start 2024-04-22T07:04:39.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.320+0000 7f0cf0231640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ce80ce380 0x7f0ce80ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.320+0000 7f0cf0231640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ce8161350 0x7f0ce8163740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.320+0000 7f0cf0231640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ce80c54a0 con 0x7f0ce8070db0 2024-04-22T07:04:39.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.320+0000 7f0cf0231640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f0ce80ca4d0 con 0x7f0ce8161350 2024-04-22T07:04:39.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.320+0000 7f0cf0231640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0ce80c8e90 con 0x7f0ce80ce380 2024-04-22T07:04:39.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.320+0000 7f0cee7ce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ce8161350 0x7f0ce8163740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.320+0000 7f0ced7cc640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0ce8070db0 0x7f0ce80c3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48292/0 (socket says 172.21.15.92:48292) 2024-04-22T07:04:39.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.320+0000 7f0cee7ce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ce8161350 0x7f0ce8163740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51388/0 (socket says 172.21.15.92:51388) 2024-04-22T07:04:39.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.320+0000 7f0ced7cc640 1 -- 172.21.15.92:0/736277874 learned_addr learned my addr 172.21.15.92:0/736277874 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:39.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.321+0000 7f0cedfcd640 1 --2- 172.21.15.92:0/736277874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ce80ce380 0x7f0ce80ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.321+0000 7f0cee7ce640 1 -- 172.21.15.92:0/736277874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ce80ce380 msgr2=0x7f0ce80ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:39.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.321+0000 7f0cee7ce640 1 --2- 172.21.15.92:0/736277874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ce80ce380 0x7f0ce80ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.321+0000 7f0cee7ce640 1 -- 172.21.15.92:0/736277874 >> v1:172.21.15.92:6789/0 conn(0x7f0ce8070db0 legacy=0x7f0ce80c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.321+0000 7f0cee7ce640 1 -- 172.21.15.92:0/736277874 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce80c9170 con 0x7f0ce8161350 2024-04-22T07:04:39.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.321+0000 7f0cecfcb640 1 -- 172.21.15.92:0/736277874 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1042325106 0 0) 0x7f0ce80c54a0 con 0x7f0ce8070db0 2024-04-22T07:04:39.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.321+0000 7f0cee7ce640 1 --2- 172.21.15.92:0/736277874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce8161350 0x7f0ce8163740 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00097a0 tx=0x7f0ce002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5463093313407b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:39.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.321+0000 7f0cecfcb640 1 -- 172.21.15.92:0/736277874 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00315a0 con 0x7f0ce8161350 2024-04-22T07:04:39.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.322+0000 7f0cecfcb640 1 -- 172.21.15.92:0/736277874 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce0037070 con 0x7f0ce8161350 2024-04-22T07:04:39.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.322+0000 7f0cecfcb640 1 -- 172.21.15.92:0/736277874 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0031960 con 0x7f0ce8161350 2024-04-22T07:04:39.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.322+0000 7f0cf0231640 1 -- 172.21.15.92:0/736277874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce8161350 msgr2=0x7f0ce8163740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.322+0000 7f0cf0231640 1 --2- 172.21.15.92:0/736277874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce8161350 0x7f0ce8163740 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00097a0 tx=0x7f0ce002b840 comp rx=0 tx=0).stop 2024-04-22T07:04:39.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.322+0000 7f0cf0231640 1 -- 172.21.15.92:0/736277874 shutdown_connections 2024-04-22T07:04:39.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.322+0000 7f0cf0231640 1 --2- 172.21.15.92:0/736277874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce8161350 0x7f0ce8163740 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.322+0000 7f0cf0231640 1 --2- 172.21.15.92:0/736277874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ce80ce380 0x7f0ce80ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.322+0000 7f0cf0231640 1 -- 172.21.15.92:0/736277874 >> 172.21.15.92:0/736277874 conn(0x7f0ce806db80 msgr2=0x7f0ce806df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.322+0000 7f0cf0231640 1 -- 172.21.15.92:0/736277874 shutdown_connections 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.322+0000 7f0cf0231640 1 -- 172.21.15.92:0/736277874 wait complete. 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.323+0000 7f0cf0231640 1 Processor -- start 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.323+0000 7f0cf0231640 1 -- start start 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cf0231640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ce8070db0 0x7f0ce806cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cf0231640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce80ce380 0x7f0ce806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cf0231640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce8161350 0x7f0ce81f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cf0231640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce8070260 con 0x7f0ce8161350 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cf0231640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce8159740 con 0x7f0ce80ce380 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cf0231640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0ce80c5190 con 0x7f0ce8070db0 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cee7ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce8161350 0x7f0ce81f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cee7ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce8161350 0x7f0ce81f44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51392/0 (socket says 172.21.15.92:51392) 2024-04-22T07:04:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cee7ce640 1 -- 172.21.15.92:0/1401857694 learned_addr learned my addr 172.21.15.92:0/1401857694 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cedfcd640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ce8070db0 0x7f0ce806cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0ced7cc640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce80ce380 0x7f0ce806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cee7ce640 1 -- 172.21.15.92:0/1401857694 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ce8070db0 msgr2=0x7f0ce806cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cee7ce640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ce8070db0 0x7f0ce806cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cee7ce640 1 -- 172.21.15.92:0/1401857694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce80ce380 msgr2=0x7f0ce806d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cee7ce640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce80ce380 0x7f0ce806d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0cee7ce640 1 -- 172.21.15.92:0/1401857694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce8157e90 con 0x7f0ce8161350 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.324+0000 7f0ced7cc640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce80ce380 0x7f0ce806d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.325+0000 7f0cee7ce640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce8161350 0x7f0ce81f44a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00097a0 tx=0x7f0ce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.325+0000 7f0cde7fc640 1 -- 172.21.15.92:0/1401857694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0004230 con 0x7f0ce8161350 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.325+0000 7f0cde7fc640 1 -- 172.21.15.92:0/1401857694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce00043d0 con 0x7f0ce8161350 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.325+0000 7f0cde7fc640 1 -- 172.21.15.92:0/1401857694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0003850 con 0x7f0ce8161350 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.325+0000 7f0cf0231640 1 -- 172.21.15.92:0/1401857694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce806bfc0 con 0x7f0ce8161350 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.325+0000 7f0cf0231640 1 -- 172.21.15.92:0/1401857694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce806c580 con 0x7f0ce8161350 2024-04-22T07:04:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.326+0000 7f0cf0231640 1 -- 172.21.15.92:0/1401857694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce8068b00 con 0x7f0ce8161350 2024-04-22T07:04:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.326+0000 7f0cde7fc640 1 -- 172.21.15.92:0/1401857694 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ce00039f0 con 0x7f0ce8161350 2024-04-22T07:04:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.326+0000 7f0cde7fc640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0ccc06c8f0 0x7f0ccc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.326+0000 7f0cde7fc640 1 -- 172.21.15.92:0/1401857694 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0ce002f030 con 0x7f0ce8161350 2024-04-22T07:04:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.326+0000 7f0cedfcd640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0ccc06c8f0 0x7f0ccc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.327+0000 7f0cedfcd640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0ccc06c8f0 0x7f0ccc06edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40099c0 tx=0x7f0ce4003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.331+0000 7f0cde7fc640 1 -- 172.21.15.92:0/1401857694 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ce0041070 con 0x7f0ce8161350 2024-04-22T07:04:39.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.505+0000 7f0cf0231640 1 -- 172.21.15.92:0/1401857694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ce8152620 con 0x7f0ce8161350 2024-04-22T07:04:39.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.506+0000 7f0cde7fc640 1 -- 172.21.15.92:0/1401857694 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f0ce0044340 con 0x7f0ce8161350 2024-04-22T07:04:39.504 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-04-22T07:04:39.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.507+0000 7f0cf0231640 1 -- 172.21.15.92:0/1401857694 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0ccc06c8f0 msgr2=0x7f0ccc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.507+0000 7f0cf0231640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0ccc06c8f0 0x7f0ccc06edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40099c0 tx=0x7f0ce4003990 comp rx=0 tx=0).stop 2024-04-22T07:04:39.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.507+0000 7f0cf0231640 1 -- 172.21.15.92:0/1401857694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce8161350 msgr2=0x7f0ce81f44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.507+0000 7f0cf0231640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce8161350 0x7f0ce81f44a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00097a0 tx=0x7f0ce00046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.507+0000 7f0cf0231640 1 -- 172.21.15.92:0/1401857694 shutdown_connections 2024-04-22T07:04:39.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.507+0000 7f0cf0231640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0ccc06c8f0 0x7f0ccc06edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.507+0000 7f0cf0231640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce8161350 0x7f0ce81f44a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.507+0000 7f0cf0231640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ce80ce380 0x7f0ce806d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.507+0000 7f0cf0231640 1 --2- 172.21.15.92:0/1401857694 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ce8070db0 0x7f0ce806cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.507+0000 7f0cf0231640 1 -- 172.21.15.92:0/1401857694 >> 172.21.15.92:0/1401857694 conn(0x7f0ce806db80 msgr2=0x7f0ce80cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:39.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.508+0000 7f0cf0231640 1 -- 172.21.15.92:0/1401857694 shutdown_connections 2024-04-22T07:04:39.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.508+0000 7f0cf0231640 1 -- 172.21.15.92:0/1401857694 wait complete. 2024-04-22T07:04:39.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:04:39.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.618+0000 7f8eeb72b640 1 Processor -- start 2024-04-22T07:04:39.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.618+0000 7f8eeb72b640 1 -- start start 2024-04-22T07:04:39.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8eeb72b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8ee4151d90 0x7f8ee40c5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8eeb72b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ee406eed0 0x7f8ee40c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8eeb72b640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ee40c88b0 con 0x7f8ee40c6c20 2024-04-22T07:04:39.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8eeb72b640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f8ee4152170 con 0x7f8ee406eed0 2024-04-22T07:04:39.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8eeb72b640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f8ee40c7680 con 0x7f8ee4151d90 2024-04-22T07:04:39.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8ee8cc6640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ee406eed0 0x7f8ee40c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8ee8cc6640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ee406eed0 0x7f8ee40c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51404/0 (socket says 172.21.15.92:51404) 2024-04-22T07:04:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8ee8cc6640 1 -- 172.21.15.92:0/1088199398 learned_addr learned my addr 172.21.15.92:0/1088199398 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8ee94c7640 1 --2- 172.21.15.92:0/1088199398 >> v2:172.21.15.145:3300/0 conn(0x7f8ee4151d90 0x7f8ee40c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8ee8cc6640 1 -- 172.21.15.92:0/1088199398 >> v2:172.21.15.145:3300/0 conn(0x7f8ee4151d90 msgr2=0x7f8ee40c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8ee8cc6640 1 --2- 172.21.15.92:0/1088199398 >> v2:172.21.15.145:3300/0 conn(0x7f8ee4151d90 0x7f8ee40c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8ee8cc6640 1 -- 172.21.15.92:0/1088199398 >> v1:172.21.15.145:6789/0 conn(0x7f8ee40c6c20 legacy=0x7f8ee4157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.619+0000 7f8ee8cc6640 1 -- 172.21.15.92:0/1088199398 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee40c73a0 con 0x7f8ee406eed0 2024-04-22T07:04:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.620+0000 7f8ee8cc6640 1 --2- 172.21.15.92:0/1088199398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee406eed0 0x7f8ee40c58a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40712e0 tx=0x7f8ed402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8770083d7a9c30bf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.620+0000 7f8edbfff640 1 -- 172.21.15.92:0/1088199398 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4034070 con 0x7f8ee406eed0 2024-04-22T07:04:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.620+0000 7f8edbfff640 1 -- 172.21.15.92:0/1088199398 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed4030860 con 0x7f8ee406eed0 2024-04-22T07:04:39.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.620+0000 7f8edbfff640 1 -- 172.21.15.92:0/1088199398 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4030c30 con 0x7f8ee406eed0 2024-04-22T07:04:39.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.621+0000 7f8eeb72b640 1 -- 172.21.15.92:0/1088199398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee406eed0 msgr2=0x7f8ee40c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.621+0000 7f8eeb72b640 1 --2- 172.21.15.92:0/1088199398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee406eed0 0x7f8ee40c58a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40712e0 tx=0x7f8ed402b9a0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.621+0000 7f8eeb72b640 1 -- 172.21.15.92:0/1088199398 shutdown_connections 2024-04-22T07:04:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.621+0000 7f8eeb72b640 1 --2- 172.21.15.92:0/1088199398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee406eed0 0x7f8ee40c58a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.621+0000 7f8eeb72b640 1 --2- 172.21.15.92:0/1088199398 >> v2:172.21.15.145:3300/0 conn(0x7f8ee4151d90 0x7f8ee40c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.621+0000 7f8eeb72b640 1 -- 172.21.15.92:0/1088199398 >> 172.21.15.92:0/1088199398 conn(0x7f8ee40cc220 msgr2=0x7f8ee40cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.621+0000 7f8eeb72b640 1 -- 172.21.15.92:0/1088199398 shutdown_connections 2024-04-22T07:04:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.621+0000 7f8eeb72b640 1 -- 172.21.15.92:0/1088199398 wait complete. 2024-04-22T07:04:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.622+0000 7f8eeb72b640 1 Processor -- start 2024-04-22T07:04:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.622+0000 7f8eeb72b640 1 -- start start 2024-04-22T07:04:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8eeb72b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ee406eed0 0x7f8ee4071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8eeb72b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee40c6c20 0x7f8ee414c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8eeb72b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8ee4151d90 0x7f8ee4143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8eeb72b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee40c3ba0 con 0x7f8ee40c6c20 2024-04-22T07:04:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8eeb72b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee40cd160 con 0x7f8ee406eed0 2024-04-22T07:04:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8eeb72b640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee41582b0 con 0x7f8ee4151d90 2024-04-22T07:04:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee8cc6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee40c6c20 0x7f8ee414c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee8cc6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee40c6c20 0x7f8ee414c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51406/0 (socket says 172.21.15.92:51406) 2024-04-22T07:04:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee8cc6640 1 -- 172.21.15.92:0/964207148 learned_addr learned my addr 172.21.15.92:0/964207148 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee9cc8640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8ee4151d90 0x7f8ee4143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee94c7640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ee406eed0 0x7f8ee4071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee8cc6640 1 -- 172.21.15.92:0/964207148 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8ee4151d90 msgr2=0x7f8ee4143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee8cc6640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8ee4151d90 0x7f8ee4143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee8cc6640 1 -- 172.21.15.92:0/964207148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ee406eed0 msgr2=0x7f8ee4071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee8cc6640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ee406eed0 0x7f8ee4071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee8cc6640 1 -- 172.21.15.92:0/964207148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee41f1630 con 0x7f8ee40c6c20 2024-04-22T07:04:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee9cc8640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8ee4151d90 0x7f8ee4143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee94c7640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ee406eed0 0x7f8ee4071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ee8cc6640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee40c6c20 0x7f8ee414c8d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8ed402b9e0 tx=0x7f8ed4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ed9ffb640 1 -- 172.21.15.92:0/964207148 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4048070 con 0x7f8ee40c6c20 2024-04-22T07:04:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ed9ffb640 1 -- 172.21.15.92:0/964207148 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed4030660 con 0x7f8ee40c6c20 2024-04-22T07:04:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8ed9ffb640 1 -- 172.21.15.92:0/964207148 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4034040 con 0x7f8ee40c6c20 2024-04-22T07:04:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.623+0000 7f8eeb72b640 1 -- 172.21.15.92:0/964207148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee41f18d0 con 0x7f8ee40c6c20 2024-04-22T07:04:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.624+0000 7f8eeb72b640 1 -- 172.21.15.92:0/964207148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee41f1e90 con 0x7f8ee40c6c20 2024-04-22T07:04:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.625+0000 7f8eeb72b640 1 -- 172.21.15.92:0/964207148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee4069930 con 0x7f8ee40c6c20 2024-04-22T07:04:39.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.625+0000 7f8ed9ffb640 1 -- 172.21.15.92:0/964207148 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ed404b070 con 0x7f8ee40c6c20 2024-04-22T07:04:39.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.625+0000 7f8ed9ffb640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8ec806c8f0 0x7f8ec806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.625+0000 7f8ed9ffb640 1 -- 172.21.15.92:0/964207148 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8ed40adba0 con 0x7f8ee40c6c20 2024-04-22T07:04:39.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.625+0000 7f8ee94c7640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8ec806c8f0 0x7f8ec806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.626+0000 7f8ee94c7640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8ec806c8f0 0x7f8ec806edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0006fd0 tx=0x7f8ee0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:39.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.630+0000 7f8ed9ffb640 1 -- 172.21.15.92:0/964207148 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ed407c060 con 0x7f8ee40c6c20 2024-04-22T07:04:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.807+0000 7f8eeb72b640 1 -- 172.21.15.92:0/964207148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8ee40c6610 con 0x7f8ee40c6c20 2024-04-22T07:04:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.807+0000 7f8ed9ffb640 1 -- 172.21.15.92:0/964207148 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f8ed407b960 con 0x7f8ee40c6c20 2024-04-22T07:04:39.806 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-04-22T07:04:39.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.809+0000 7f8eeb72b640 1 -- 172.21.15.92:0/964207148 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8ec806c8f0 msgr2=0x7f8ec806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.809+0000 7f8eeb72b640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8ec806c8f0 0x7f8ec806edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0006fd0 tx=0x7f8ee0008040 comp rx=0 tx=0).stop 2024-04-22T07:04:39.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.809+0000 7f8eeb72b640 1 -- 172.21.15.92:0/964207148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee40c6c20 msgr2=0x7f8ee414c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.809+0000 7f8eeb72b640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee40c6c20 0x7f8ee414c8d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8ed402b9e0 tx=0x7f8ed4004730 comp rx=0 tx=0).stop 2024-04-22T07:04:39.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.809+0000 7f8eeb72b640 1 -- 172.21.15.92:0/964207148 shutdown_connections 2024-04-22T07:04:39.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.809+0000 7f8eeb72b640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8ec806c8f0 0x7f8ec806edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.809+0000 7f8eeb72b640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8ee4151d90 0x7f8ee4143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.809+0000 7f8eeb72b640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee40c6c20 0x7f8ee414c8d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.809+0000 7f8eeb72b640 1 --2- 172.21.15.92:0/964207148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ee406eed0 0x7f8ee4071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.809+0000 7f8eeb72b640 1 -- 172.21.15.92:0/964207148 >> 172.21.15.92:0/964207148 conn(0x7f8ee40cc220 msgr2=0x7f8ee41500e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.809+0000 7f8eeb72b640 1 -- 172.21.15.92:0/964207148 shutdown_connections 2024-04-22T07:04:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.809+0000 7f8eeb72b640 1 -- 172.21.15.92:0/964207148 wait complete. 2024-04-22T07:04:39.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:04:39.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.920+0000 7fcca765c640 1 Processor -- start 2024-04-22T07:04:39.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.920+0000 7fcca765c640 1 -- start start 2024-04-22T07:04:39.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca765c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcca0158400 0x7fcca00ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca765c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcca006d6c0 0x7fcca00ca490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca765c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcca0070e90 0x7fcca00ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca765c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fcca01587e0 con 0x7fcca006d6c0 2024-04-22T07:04:39.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca765c640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcca006bcd0 con 0x7fcca0158400 2024-04-22T07:04:39.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca765c640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcca006bfb0 con 0x7fcca0070e90 2024-04-22T07:04:39.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca4bf7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcca006d6c0 0x7fcca00ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca4bf7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcca006d6c0 0x7fcca00ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51416/0 (socket says 172.21.15.92:51416) 2024-04-22T07:04:39.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca4bf7640 1 -- 172.21.15.92:0/1961004695 learned_addr learned my addr 172.21.15.92:0/1961004695 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:39.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca53f8640 1 --2- 172.21.15.92:0/1961004695 >> v2:172.21.15.145:3300/0 conn(0x7fcca0158400 0x7fcca00ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca5bf9640 1 --2- 172.21.15.92:0/1961004695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcca0070e90 0x7fcca00ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca4bf7640 1 -- 172.21.15.92:0/1961004695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcca0070e90 msgr2=0x7fcca00ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:39.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca4bf7640 1 --2- 172.21.15.92:0/1961004695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcca0070e90 0x7fcca00ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca4bf7640 1 -- 172.21.15.92:0/1961004695 >> v2:172.21.15.145:3300/0 conn(0x7fcca0158400 msgr2=0x7fcca00ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:39.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca4bf7640 1 --2- 172.21.15.92:0/1961004695 >> v2:172.21.15.145:3300/0 conn(0x7fcca0158400 0x7fcca00ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca4bf7640 1 -- 172.21.15.92:0/1961004695 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca0070130 con 0x7fcca006d6c0 2024-04-22T07:04:39.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca53f8640 1 --2- 172.21.15.92:0/1961004695 >> v2:172.21.15.145:3300/0 conn(0x7fcca0158400 0x7fcca00ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:39.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.921+0000 7fcca5bf9640 1 --2- 172.21.15.92:0/1961004695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcca0070e90 0x7fcca00ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:39.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.922+0000 7fcca4bf7640 1 --2- 172.21.15.92:0/1961004695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca006d6c0 0x7fcca00ca490 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcca00b1470 tx=0x7fcc9002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7478f4390ff18331 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:39.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.922+0000 7fcc97fff640 1 -- 172.21.15.92:0/1961004695 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90034070 con 0x7fcca006d6c0 2024-04-22T07:04:39.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.922+0000 7fcc97fff640 1 -- 172.21.15.92:0/1961004695 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc90030860 con 0x7fcca006d6c0 2024-04-22T07:04:39.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.922+0000 7fcc97fff640 1 -- 172.21.15.92:0/1961004695 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90030c30 con 0x7fcca006d6c0 2024-04-22T07:04:39.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.922+0000 7fcca765c640 1 -- 172.21.15.92:0/1961004695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca006d6c0 msgr2=0x7fcca00ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.922+0000 7fcca765c640 1 --2- 172.21.15.92:0/1961004695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca006d6c0 0x7fcca00ca490 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcca00b1470 tx=0x7fcc9002b9a0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.922+0000 7fcca765c640 1 -- 172.21.15.92:0/1961004695 shutdown_connections 2024-04-22T07:04:39.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.922+0000 7fcca765c640 1 --2- 172.21.15.92:0/1961004695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcca0070e90 0x7fcca00ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.922+0000 7fcca765c640 1 --2- 172.21.15.92:0/1961004695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca006d6c0 0x7fcca00ca490 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.922+0000 7fcca765c640 1 --2- 172.21.15.92:0/1961004695 >> v2:172.21.15.145:3300/0 conn(0x7fcca0158400 0x7fcca00ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.922+0000 7fcca765c640 1 -- 172.21.15.92:0/1961004695 >> 172.21.15.92:0/1961004695 conn(0x7fcca006db80 msgr2=0x7fcca006df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:39.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.923+0000 7fcca765c640 1 -- 172.21.15.92:0/1961004695 shutdown_connections 2024-04-22T07:04:39.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.923+0000 7fcca765c640 1 -- 172.21.15.92:0/1961004695 wait complete. 2024-04-22T07:04:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.924+0000 7fcca765c640 1 Processor -- start 2024-04-22T07:04:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.924+0000 7fcca765c640 1 -- start start 2024-04-22T07:04:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.924+0000 7fcca765c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcca006d6c0 0x7fcca01f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.924+0000 7fcca765c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca0070e90 0x7fcca015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.924+0000 7fcca765c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcca0158400 0x7fcca015c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.924+0000 7fcca765c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcca015e310 con 0x7fcca0070e90 2024-04-22T07:04:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.924+0000 7fcca765c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcca00706f0 con 0x7fcca006d6c0 2024-04-22T07:04:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.924+0000 7fcca765c640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcca00709d0 con 0x7fcca0158400 2024-04-22T07:04:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.924+0000 7fcca4bf7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca0070e90 0x7fcca015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.924+0000 7fcca4bf7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca0070e90 0x7fcca015bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51428/0 (socket says 172.21.15.92:51428) 2024-04-22T07:04:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.924+0000 7fcca4bf7640 1 -- 172.21.15.92:0/2415128615 learned_addr learned my addr 172.21.15.92:0/2415128615 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcca53f8640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcca006d6c0 0x7fcca01f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcca5bf9640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcca0158400 0x7fcca015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcca4bf7640 1 -- 172.21.15.92:0/2415128615 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcca0158400 msgr2=0x7fcca015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcca4bf7640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcca0158400 0x7fcca015c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcca4bf7640 1 -- 172.21.15.92:0/2415128615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcca006d6c0 msgr2=0x7fcca01f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcca4bf7640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcca006d6c0 0x7fcca01f3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcca4bf7640 1 -- 172.21.15.92:0/2415128615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca01f45a0 con 0x7fcca0070e90 2024-04-22T07:04:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcca53f8640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcca006d6c0 0x7fcca01f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcca4bf7640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca0070e90 0x7fcca015bf00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcc90000c00 tx=0x7fcc90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcc95ffb640 1 -- 172.21.15.92:0/2415128615 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90048070 con 0x7fcca0070e90 2024-04-22T07:04:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcc95ffb640 1 -- 172.21.15.92:0/2415128615 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc90030660 con 0x7fcca0070e90 2024-04-22T07:04:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcc95ffb640 1 -- 172.21.15.92:0/2415128615 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90034040 con 0x7fcca0070e90 2024-04-22T07:04:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcca765c640 1 -- 172.21.15.92:0/2415128615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca01f4880 con 0x7fcca0070e90 2024-04-22T07:04:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.925+0000 7fcca765c640 1 -- 172.21.15.92:0/2415128615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca015db50 con 0x7fcca0070e90 2024-04-22T07:04:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.926+0000 7fcca765c640 1 -- 172.21.15.92:0/2415128615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca01589e0 con 0x7fcca0070e90 2024-04-22T07:04:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.927+0000 7fcc95ffb640 1 -- 172.21.15.92:0/2415128615 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc9004b070 con 0x7fcca0070e90 2024-04-22T07:04:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.927+0000 7fcc95ffb640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcc8406c8f0 0x7fcc8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.927+0000 7fcc95ffb640 1 -- 172.21.15.92:0/2415128615 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcc900adc00 con 0x7fcca0070e90 2024-04-22T07:04:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.927+0000 7fcca53f8640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcc8406c8f0 0x7fcc8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.928+0000 7fcca53f8640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcc8406c8f0 0x7fcc8406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c006fd0 tx=0x7fcc9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:39.932+0000 7fcc95ffb640 1 -- 172.21.15.92:0/2415128615 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcca01589e0 con 0x7fcca0070e90 2024-04-22T07:04:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.109+0000 7fcca765c640 1 -- 172.21.15.92:0/2415128615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcca0069930 con 0x7fcca0070e90 2024-04-22T07:04:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.109+0000 7fcc95ffb640 1 -- 172.21.15.92:0/2415128615 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fcc900b4080 con 0x7fcca0070e90 2024-04-22T07:04:40.107 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-04-22T07:04:40.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.112+0000 7fcca765c640 1 -- 172.21.15.92:0/2415128615 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcc8406c8f0 msgr2=0x7fcc8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.112+0000 7fcca765c640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcc8406c8f0 0x7fcc8406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c006fd0 tx=0x7fcc9c008040 comp rx=0 tx=0).stop 2024-04-22T07:04:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.112+0000 7fcca765c640 1 -- 172.21.15.92:0/2415128615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca0070e90 msgr2=0x7fcca015bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.112+0000 7fcca765c640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca0070e90 0x7fcca015bf00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcc90000c00 tx=0x7fcc90004730 comp rx=0 tx=0).stop 2024-04-22T07:04:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.113+0000 7fcca765c640 1 -- 172.21.15.92:0/2415128615 shutdown_connections 2024-04-22T07:04:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.113+0000 7fcca765c640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcc8406c8f0 0x7fcc8406edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.113+0000 7fcca765c640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcca0158400 0x7fcca015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.113+0000 7fcca765c640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcca0070e90 0x7fcca015bf00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.113+0000 7fcca765c640 1 --2- 172.21.15.92:0/2415128615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcca006d6c0 0x7fcca01f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.113+0000 7fcca765c640 1 -- 172.21.15.92:0/2415128615 >> 172.21.15.92:0/2415128615 conn(0x7fcca006db80 msgr2=0x7fcca00c9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:40.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.113+0000 7fcca765c640 1 -- 172.21.15.92:0/2415128615 shutdown_connections 2024-04-22T07:04:40.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.113+0000 7fcca765c640 1 -- 172.21.15.92:0/2415128615 wait complete. 2024-04-22T07:04:40.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:40.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.220+0000 7f8df6a9a640 1 Processor -- start 2024-04-22T07:04:40.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.221+0000 7f8df6a9a640 1 -- start start 2024-04-22T07:04:40.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.221+0000 7f8df6a9a640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8df0151140 0x7f8df0156f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.221+0000 7f8df6a9a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8df00c7350 0x7f8df006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.221+0000 7f8df6a9a640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8df00ca310 con 0x7f8df00c6ec0 2024-04-22T07:04:40.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.221+0000 7f8df6a9a640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f8df00c77c0 con 0x7f8df00c7350 2024-04-22T07:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.221+0000 7f8df6a9a640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f8df00cb200 con 0x7f8df0151140 2024-04-22T07:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.221+0000 7f8df5037640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8df00c7350 0x7f8df006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.221+0000 7f8df5037640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8df00c7350 0x7f8df006faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51436/0 (socket says 172.21.15.92:51436) 2024-04-22T07:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.222+0000 7f8df5037640 1 -- 172.21.15.92:0/2063734803 learned_addr learned my addr 172.21.15.92:0/2063734803 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.222+0000 7f8df4836640 1 --2- 172.21.15.92:0/2063734803 >> v2:172.21.15.145:3300/0 conn(0x7f8df0151140 0x7f8df0156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.222+0000 7f8df5037640 1 -- 172.21.15.92:0/2063734803 >> v2:172.21.15.145:3300/0 conn(0x7f8df0151140 msgr2=0x7f8df0156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.222+0000 7f8df5037640 1 --2- 172.21.15.92:0/2063734803 >> v2:172.21.15.145:3300/0 conn(0x7f8df0151140 0x7f8df0156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.222+0000 7f8df5037640 1 -- 172.21.15.92:0/2063734803 >> v1:172.21.15.145:6789/0 conn(0x7f8df00c6ec0 legacy=0x7f8df006f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.222+0000 7f8df5037640 1 -- 172.21.15.92:0/2063734803 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df00ca5f0 con 0x7f8df00c7350 2024-04-22T07:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.222+0000 7f8df4836640 1 --2- 172.21.15.92:0/2063734803 >> v2:172.21.15.145:3300/0 conn(0x7f8df0151140 0x7f8df0156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.222+0000 7f8df5037640 1 --2- 172.21.15.92:0/2063734803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df00c7350 0x7f8df006faa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8df0068250 tx=0x7f8de802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fbf8995510385a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.223+0000 7f8def7fe640 1 -- 172.21.15.92:0/2063734803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de802cb20 con 0x7f8df00c7350 2024-04-22T07:04:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.223+0000 7f8def7fe640 1 -- 172.21.15.92:0/2063734803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de802ccc0 con 0x7f8df00c7350 2024-04-22T07:04:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.223+0000 7f8def7fe640 1 -- 172.21.15.92:0/2063734803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de8034610 con 0x7f8df00c7350 2024-04-22T07:04:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.223+0000 7f8df6a9a640 1 -- 172.21.15.92:0/2063734803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df00c7350 msgr2=0x7f8df006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.223+0000 7f8df6a9a640 1 --2- 172.21.15.92:0/2063734803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df00c7350 0x7f8df006faa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8df0068250 tx=0x7f8de802bf80 comp rx=0 tx=0).stop 2024-04-22T07:04:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.223+0000 7f8df6a9a640 1 -- 172.21.15.92:0/2063734803 shutdown_connections 2024-04-22T07:04:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.223+0000 7f8df6a9a640 1 --2- 172.21.15.92:0/2063734803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df00c7350 0x7f8df006faa0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.223+0000 7f8df6a9a640 1 --2- 172.21.15.92:0/2063734803 >> v2:172.21.15.145:3300/0 conn(0x7f8df0151140 0x7f8df0156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.223+0000 7f8df6a9a640 1 -- 172.21.15.92:0/2063734803 >> 172.21.15.92:0/2063734803 conn(0x7f8df00cc220 msgr2=0x7f8df00cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:40.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.223+0000 7f8df6a9a640 1 -- 172.21.15.92:0/2063734803 shutdown_connections 2024-04-22T07:04:40.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.223+0000 7f8df6a9a640 1 -- 172.21.15.92:0/2063734803 wait complete. 2024-04-22T07:04:40.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.224+0000 7f8df6a9a640 1 Processor -- start 2024-04-22T07:04:40.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.224+0000 7f8df6a9a640 1 -- start start 2024-04-22T07:04:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.225+0000 7f8df6a9a640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8df00c6ec0 0x7f8df0158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.225+0000 7f8df6a9a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8df00c7350 0x7f8df013e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.225+0000 7f8df6a9a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df0151140 0x7f8df00730d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.225+0000 7f8df6a9a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8df00cae90 con 0x7f8df0151140 2024-04-22T07:04:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.225+0000 7f8df6a9a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8df00cabb0 con 0x7f8df00c7350 2024-04-22T07:04:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.225+0000 7f8df6a9a640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8df00ca020 con 0x7f8df00c6ec0 2024-04-22T07:04:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.225+0000 7f8df5037640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df0151140 0x7f8df00730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.225+0000 7f8df5037640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df0151140 0x7f8df00730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51440/0 (socket says 172.21.15.92:51440) 2024-04-22T07:04:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.225+0000 7f8df5037640 1 -- 172.21.15.92:0/980715345 learned_addr learned my addr 172.21.15.92:0/980715345 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.225+0000 7f8df4836640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8df00c6ec0 0x7f8df0158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.225+0000 7f8deffff640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8df00c7350 0x7f8df013e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.226+0000 7f8df5037640 1 -- 172.21.15.92:0/980715345 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8df00c6ec0 msgr2=0x7f8df0158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.226+0000 7f8df5037640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8df00c6ec0 0x7f8df0158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.226+0000 7f8df5037640 1 -- 172.21.15.92:0/980715345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8df00c7350 msgr2=0x7f8df013e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.226+0000 7f8df5037640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8df00c7350 0x7f8df013e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.226+0000 7f8df5037640 1 -- 172.21.15.92:0/980715345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df0154020 con 0x7f8df0151140 2024-04-22T07:04:40.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.226+0000 7f8df5037640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df0151140 0x7f8df00730d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8de8036040 tx=0x7f8de8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.226+0000 7f8ded7fa640 1 -- 172.21.15.92:0/980715345 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de8043020 con 0x7f8df0151140 2024-04-22T07:04:40.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.226+0000 7f8ded7fa640 1 -- 172.21.15.92:0/980715345 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de803e450 con 0x7f8df0151140 2024-04-22T07:04:40.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.226+0000 7f8df6a9a640 1 -- 172.21.15.92:0/980715345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df0154300 con 0x7f8df0151140 2024-04-22T07:04:40.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.226+0000 7f8ded7fa640 1 -- 172.21.15.92:0/980715345 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de803e820 con 0x7f8df0151140 2024-04-22T07:04:40.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.226+0000 7f8df6a9a640 1 -- 172.21.15.92:0/980715345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df013dd40 con 0x7f8df0151140 2024-04-22T07:04:40.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.227+0000 7f8df6a9a640 1 -- 172.21.15.92:0/980715345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df0070280 con 0x7f8df0151140 2024-04-22T07:04:40.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.227+0000 7f8ded7fa640 1 -- 172.21.15.92:0/980715345 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8de8046020 con 0x7f8df0151140 2024-04-22T07:04:40.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.228+0000 7f8ded7fa640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8dd406c8f0 0x7f8dd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.228+0000 7f8ded7fa640 1 -- 172.21.15.92:0/980715345 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8de80ae610 con 0x7f8df0151140 2024-04-22T07:04:40.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.228+0000 7f8df4836640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8dd406c8f0 0x7f8dd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.228+0000 7f8df4836640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8dd406c8f0 0x7f8dd406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8de400aac0 tx=0x7f8de4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.232+0000 7f8ded7fa640 1 -- 172.21.15.92:0/980715345 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8df0070280 con 0x7f8df0151140 2024-04-22T07:04:40.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.409+0000 7f8df6a9a640 1 -- 172.21.15.92:0/980715345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8df013d850 con 0x7f8df0151140 2024-04-22T07:04:40.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.410+0000 7f8ded7fa640 1 -- 172.21.15.92:0/980715345 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f8de804d020 con 0x7f8df0151140 2024-04-22T07:04:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:40.408 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:04:04.784969+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:40.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.412+0000 7f8df6a9a640 1 -- 172.21.15.92:0/980715345 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8dd406c8f0 msgr2=0x7f8dd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.412+0000 7f8df6a9a640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8dd406c8f0 0x7f8dd406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8de400aac0 tx=0x7f8de4008040 comp rx=0 tx=0).stop 2024-04-22T07:04:40.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.412+0000 7f8df6a9a640 1 -- 172.21.15.92:0/980715345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df0151140 msgr2=0x7f8df00730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.412+0000 7f8df6a9a640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df0151140 0x7f8df00730d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8de8036040 tx=0x7f8de8004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:40.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.412+0000 7f8df6a9a640 1 -- 172.21.15.92:0/980715345 shutdown_connections 2024-04-22T07:04:40.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.412+0000 7f8df6a9a640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8dd406c8f0 0x7f8dd406edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.412+0000 7f8df6a9a640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8df0151140 0x7f8df00730d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.412+0000 7f8df6a9a640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8df00c7350 0x7f8df013e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.412+0000 7f8df6a9a640 1 --2- 172.21.15.92:0/980715345 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8df00c6ec0 0x7f8df0158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.412+0000 7f8df6a9a640 1 -- 172.21.15.92:0/980715345 >> 172.21.15.92:0/980715345 conn(0x7f8df00cc220 msgr2=0x7f8df00c8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:40.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.412+0000 7f8df6a9a640 1 -- 172.21.15.92:0/980715345 shutdown_connections 2024-04-22T07:04:40.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.412+0000 7f8df6a9a640 1 -- 172.21.15.92:0/980715345 wait complete. 2024-04-22T07:04:40.421 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:03:57.947601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:01.058111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:02.085602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:40.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.523+0000 7f682554b640 1 Processor -- start 2024-04-22T07:04:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.523+0000 7f682554b640 1 -- start start 2024-04-22T07:04:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.524+0000 7f682554b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6820156220 0x7f6820070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.524+0000 7f682554b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68200c43c0 0x7f6820159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.524+0000 7f682554b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68200c3410 con 0x7f68200c8df0 2024-04-22T07:04:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.524+0000 7f682554b640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f682006ffe0 con 0x7f6820156220 2024-04-22T07:04:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.524+0000 7f682554b640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f68200c3990 con 0x7f68200c43c0 2024-04-22T07:04:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.524+0000 7f6817fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f68200c8df0 0x7f68200c3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48298/0 (socket says 172.21.15.92:48298) 2024-04-22T07:04:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.524+0000 7f6817fff640 1 -- 172.21.15.92:0/481196940 learned_addr learned my addr 172.21.15.92:0/481196940 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681effd640 1 --2- 172.21.15.92:0/481196940 >> v2:172.21.15.145:3300/0 conn(0x7f6820156220 0x7f6820070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681f7fe640 1 --2- 172.21.15.92:0/481196940 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68200c43c0 0x7f6820159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 156869412 0 0) 0x7f68200c3410 con 0x7f68200c8df0 2024-04-22T07:04:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f680c003520 con 0x7f68200c8df0 2024-04-22T07:04:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1179925446 0 0) 0x7f680c003520 con 0x7f68200c8df0 2024-04-22T07:04:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68200c43c0 msgr2=0x7f6820159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 --2- 172.21.15.92:0/481196940 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68200c43c0 0x7f6820159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 >> v2:172.21.15.145:3300/0 conn(0x7f6820156220 msgr2=0x7f6820070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 --2- 172.21.15.92:0/481196940 >> v2:172.21.15.145:3300/0 conn(0x7f6820156220 0x7f6820070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68200c36b0 con 0x7f68200c8df0 2024-04-22T07:04:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f6808001670 con 0x7f68200c8df0 2024-04-22T07:04:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 >> v1:172.21.15.92:6789/0 conn(0x7f68200c8df0 legacy=0x7f68200c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 --2- 172.21.15.92:0/481196940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680c0039b0 0x7f680c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 --2- 172.21.15.92:0/481196940 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f680c004800 0x7f680c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 --2- 172.21.15.92:0/481196940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c005180 0x7f680c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f682006ffe0 con 0x7f680c005180 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f68200c3990 con 0x7f680c0039b0 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f680c004630 con 0x7f680c004800 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681f7fe640 1 --2- 172.21.15.92:0/481196940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c005180 0x7f680c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681effd640 1 --2- 172.21.15.92:0/481196940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680c0039b0 0x7f680c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f6817fff640 1 --2- 172.21.15.92:0/481196940 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f680c004800 0x7f680c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681f7fe640 1 -- 172.21.15.92:0/481196940 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f680c004800 msgr2=0x7f680c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681f7fe640 1 --2- 172.21.15.92:0/481196940 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f680c004800 0x7f680c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681f7fe640 1 -- 172.21.15.92:0/481196940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680c0039b0 msgr2=0x7f680c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681f7fe640 1 --2- 172.21.15.92:0/481196940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680c0039b0 0x7f680c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.525+0000 7f681f7fe640 1 -- 172.21.15.92:0/481196940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c003520 con 0x7f680c005180 2024-04-22T07:04:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.526+0000 7f681f7fe640 1 --2- 172.21.15.92:0/481196940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c005180 0x7f680c00b9c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6818009f90 tx=0x7f680c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9613b4f8774fded1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.526+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818039070 con 0x7f680c005180 2024-04-22T07:04:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.526+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f681802cbd0 con 0x7f680c005180 2024-04-22T07:04:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.526+0000 7f681e7fc640 1 -- 172.21.15.92:0/481196940 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818009c10 con 0x7f680c005180 2024-04-22T07:04:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.527+0000 7f682554b640 1 -- 172.21.15.92:0/481196940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c005180 msgr2=0x7f680c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.527+0000 7f682554b640 1 --2- 172.21.15.92:0/481196940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c005180 0x7f680c00b9c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6818009f90 tx=0x7f680c005830 comp rx=0 tx=0).stop 2024-04-22T07:04:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.527+0000 7f681f7fe640 1 -- 172.21.15.92:0/481196940 reap_dead start 2024-04-22T07:04:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.527+0000 7f682554b640 1 -- 172.21.15.92:0/481196940 shutdown_connections 2024-04-22T07:04:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.527+0000 7f682554b640 1 -- 172.21.15.92:0/481196940 >> 172.21.15.92:0/481196940 conn(0x7f682006e210 msgr2=0x7f6820071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.527+0000 7f682554b640 1 -- 172.21.15.92:0/481196940 shutdown_connections 2024-04-22T07:04:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.527+0000 7f682554b640 1 -- 172.21.15.92:0/481196940 wait complete. 2024-04-22T07:04:40.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.528+0000 7f682554b640 1 Processor -- start 2024-04-22T07:04:40.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.528+0000 7f682554b640 1 -- start start 2024-04-22T07:04:40.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f682554b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c005180 0x7f6820144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f682554b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68200ce960 0x7f6820145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f682554b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68200cef90 0x7f68200c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f682554b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f68200720c0 con 0x7f680c005180 2024-04-22T07:04:40.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f682554b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f682006fc70 con 0x7f68200ce960 2024-04-22T07:04:40.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f682554b640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6820156600 con 0x7f68200cef90 2024-04-22T07:04:40.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f681effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c005180 0x7f6820144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f681effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c005180 0x7f6820144d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51448/0 (socket says 172.21.15.92:51448) 2024-04-22T07:04:40.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f681effd640 1 -- 172.21.15.92:0/2832188403 learned_addr learned my addr 172.21.15.92:0/2832188403 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:40.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f6817fff640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68200ce960 0x7f6820145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f681f7fe640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68200cef90 0x7f68200c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f681effd640 1 -- 172.21.15.92:0/2832188403 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68200cef90 msgr2=0x7f68200c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f681effd640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68200cef90 0x7f68200c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f681effd640 1 -- 172.21.15.92:0/2832188403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68200ce960 msgr2=0x7f6820145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f681effd640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68200ce960 0x7f6820145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f681effd640 1 -- 172.21.15.92:0/2832188403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68200c85d0 con 0x7f680c005180 2024-04-22T07:04:40.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f6817fff640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68200ce960 0x7f6820145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:40.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f681effd640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c005180 0x7f6820144d30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6810011aa0 tx=0x7f6810011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f68177fe640 1 -- 172.21.15.92:0/2832188403 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681000fb80 con 0x7f680c005180 2024-04-22T07:04:40.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f68177fe640 1 -- 172.21.15.92:0/2832188403 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f681000fd20 con 0x7f680c005180 2024-04-22T07:04:40.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f682554b640 1 -- 172.21.15.92:0/2832188403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68200c68f0 con 0x7f680c005180 2024-04-22T07:04:40.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f68177fe640 1 -- 172.21.15.92:0/2832188403 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810014660 con 0x7f680c005180 2024-04-22T07:04:40.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.529+0000 7f682554b640 1 -- 172.21.15.92:0/2832188403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68200c8010 con 0x7f680c005180 2024-04-22T07:04:40.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.530+0000 7f682554b640 1 -- 172.21.15.92:0/2832188403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6820068b00 con 0x7f680c005180 2024-04-22T07:04:40.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.531+0000 7f68177fe640 1 -- 172.21.15.92:0/2832188403 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6810012040 con 0x7f680c005180 2024-04-22T07:04:40.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.531+0000 7f68177fe640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f680006c920 0x7f680006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.531+0000 7f68177fe640 1 -- 172.21.15.92:0/2832188403 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f681008c210 con 0x7f680c005180 2024-04-22T07:04:40.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.531+0000 7f6817fff640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f680006c920 0x7f680006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.532+0000 7f6817fff640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f680006c920 0x7f680006ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6808000fb0 tx=0x7f6808009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.536+0000 7f68177fe640 1 -- 172.21.15.92:0/2832188403 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6810019030 con 0x7f680c005180 2024-04-22T07:04:40.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.708+0000 7f682554b640 1 -- 172.21.15.92:0/2832188403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f68200c8840 con 0x7f680c005180 2024-04-22T07:04:40.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.709+0000 7f68177fe640 1 -- 172.21.15.92:0/2832188403 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f681005a6d0 con 0x7f680c005180 2024-04-22T07:04:40.707 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-04-22T07:04:40.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.711+0000 7f682554b640 1 -- 172.21.15.92:0/2832188403 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f680006c920 msgr2=0x7f680006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.711+0000 7f682554b640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f680006c920 0x7f680006ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6808000fb0 tx=0x7f6808009040 comp rx=0 tx=0).stop 2024-04-22T07:04:40.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.711+0000 7f682554b640 1 -- 172.21.15.92:0/2832188403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c005180 msgr2=0x7f6820144d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.711+0000 7f682554b640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c005180 0x7f6820144d30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6810011aa0 tx=0x7f6810011f60 comp rx=0 tx=0).stop 2024-04-22T07:04:40.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.711+0000 7f682554b640 1 -- 172.21.15.92:0/2832188403 shutdown_connections 2024-04-22T07:04:40.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.711+0000 7f682554b640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f680006c920 0x7f680006ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.711+0000 7f682554b640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68200cef90 0x7f68200c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.711+0000 7f682554b640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68200ce960 0x7f6820145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.711+0000 7f682554b640 1 --2- 172.21.15.92:0/2832188403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c005180 0x7f6820144d30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.711+0000 7f682554b640 1 -- 172.21.15.92:0/2832188403 >> 172.21.15.92:0/2832188403 conn(0x7f682006e210 msgr2=0x7f6820151b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:40.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.711+0000 7f682554b640 1 -- 172.21.15.92:0/2832188403 shutdown_connections 2024-04-22T07:04:40.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.711+0000 7f682554b640 1 -- 172.21.15.92:0/2832188403 wait complete. 2024-04-22T07:04:40.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:04:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.822+0000 7f86ce0eb640 1 Processor -- start 2024-04-22T07:04:40.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.822+0000 7f86ce0eb640 1 -- start start 2024-04-22T07:04:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.823+0000 7f86ce0eb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86c80ce760 0x7f86c8164a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.823+0000 7f86ce0eb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86c80c54a0 con 0x7f86c8070db0 2024-04-22T07:04:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.823+0000 7f86ce0eb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86c80c8e90 con 0x7f86c80ce380 2024-04-22T07:04:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.823+0000 7f86ce0eb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f86c80ca4d0 con 0x7f86c80ce760 2024-04-22T07:04:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.823+0000 7f86c6ffd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f86c8070db0 0x7f86c8161350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48304/0 (socket says 172.21.15.92:48304) 2024-04-22T07:04:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.823+0000 7f86c6ffd640 1 -- 172.21.15.92:0/668246381 learned_addr learned my addr 172.21.15.92:0/668246381 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c7fff640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86c80ce760 0x7f86c8164a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3177457957 0 0) 0x7f86c80c54a0 con 0x7f86c8070db0 2024-04-22T07:04:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86b0003520 con 0x7f86c8070db0 2024-04-22T07:04:40.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 180642126 0 0) 0x7f86c80c8e90 con 0x7f86c80ce380 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86c80c54a0 con 0x7f86c80ce380 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2096721515 0 0) 0x7f86b0003520 con 0x7f86c8070db0 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86c80ce760 msgr2=0x7f86c8164a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86c80ce760 0x7f86c8164a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 >> v1:172.21.15.145:6789/0 conn(0x7f86c80ce380 legacy=0x7f86c80c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c80c9170 con 0x7f86c8070db0 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f86bc001670 con 0x7f86c8070db0 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 >> v1:172.21.15.92:6789/0 conn(0x7f86c8070db0 legacy=0x7f86c8161350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86b0003b70 0x7f86b0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86b0004800 0x7f86b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00051c0 0x7f86b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86b000c000 con 0x7f86b00051c0 2024-04-22T07:04:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f86b000c1b0 con 0x7f86b0004800 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f86b000c3b0 con 0x7f86b0003b70 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c7fff640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86c80ce760 0x7f86c8164a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c6ffd640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86b0004800 0x7f86b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c77fe640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86b0003b70 0x7f86b0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c7fff640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00051c0 0x7f86b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c7fff640 1 -- 172.21.15.92:0/668246381 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86b0003b70 msgr2=0x7f86b0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c7fff640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86b0003b70 0x7f86b0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c7fff640 1 -- 172.21.15.92:0/668246381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86b0004800 msgr2=0x7f86b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c7fff640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86b0004800 0x7f86b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.824+0000 7f86c7fff640 1 -- 172.21.15.92:0/668246381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b0003520 con 0x7f86b00051c0 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.825+0000 7f86c7fff640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00051c0 0x7f86b000b9c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f86c80ceb40 tx=0x7f86b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d035f8ad4dde10b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.825+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0009d40 con 0x7f86b00051c0 2024-04-22T07:04:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.825+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c002ccf0 con 0x7f86b00051c0 2024-04-22T07:04:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.825+0000 7f86c67fc640 1 -- 172.21.15.92:0/668246381 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0003bb0 con 0x7f86b00051c0 2024-04-22T07:04:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.826+0000 7f86ce0eb640 1 -- 172.21.15.92:0/668246381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00051c0 msgr2=0x7f86b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.826+0000 7f86ce0eb640 1 --2- 172.21.15.92:0/668246381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00051c0 0x7f86b000b9c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f86c80ceb40 tx=0x7f86b0005830 comp rx=0 tx=0).stop 2024-04-22T07:04:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.826+0000 7f86c7fff640 1 -- 172.21.15.92:0/668246381 reap_dead start 2024-04-22T07:04:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.826+0000 7f86ce0eb640 1 -- 172.21.15.92:0/668246381 shutdown_connections 2024-04-22T07:04:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.826+0000 7f86ce0eb640 1 -- 172.21.15.92:0/668246381 >> 172.21.15.92:0/668246381 conn(0x7f86c806db80 msgr2=0x7f86c806df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.826+0000 7f86ce0eb640 1 -- 172.21.15.92:0/668246381 shutdown_connections 2024-04-22T07:04:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.826+0000 7f86ce0eb640 1 -- 172.21.15.92:0/668246381 wait complete. 2024-04-22T07:04:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.827+0000 7f86ce0eb640 1 Processor -- start 2024-04-22T07:04:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.827+0000 7f86ce0eb640 1 -- start start 2024-04-22T07:04:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86ce0eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c814fd70 0x7f86c8150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86ce0eb640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86c80c7720 0x7f86c80c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86ce0eb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86c80c5f10 0x7f86c81e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86ce0eb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86c8070260 con 0x7f86c814fd70 2024-04-22T07:04:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86ce0eb640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f86c8159740 con 0x7f86c80c7720 2024-04-22T07:04:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86ce0eb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f86c80cebd0 con 0x7f86c80c5f10 2024-04-22T07:04:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86c77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c814fd70 0x7f86c8150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86c77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c814fd70 0x7f86c8150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51468/0 (socket says 172.21.15.92:51468) 2024-04-22T07:04:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86c77fe640 1 -- 172.21.15.92:0/1523664102 learned_addr learned my addr 172.21.15.92:0/1523664102 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86c6ffd640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86c80c7720 0x7f86c80c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86c7fff640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86c80c5f10 0x7f86c81e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86c77fe640 1 -- 172.21.15.92:0/1523664102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86c80c5f10 msgr2=0x7f86c81e3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86c77fe640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86c80c5f10 0x7f86c81e3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86c77fe640 1 -- 172.21.15.92:0/1523664102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86c80c7720 msgr2=0x7f86c80c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86c77fe640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86c80c7720 0x7f86c80c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86c77fe640 1 -- 172.21.15.92:0/1523664102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c80c8550 con 0x7f86c814fd70 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.828+0000 7f86c77fe640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c814fd70 0x7f86c8150150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f86b800f820 tx=0x7f86b800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.829+0000 7f86affff640 1 -- 172.21.15.92:0/1523664102 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b80052e0 con 0x7f86c814fd70 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.829+0000 7f86affff640 1 -- 172.21.15.92:0/1523664102 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86b8018070 con 0x7f86c814fd70 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.829+0000 7f86ce0eb640 1 -- 172.21.15.92:0/1523664102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c81ef550 con 0x7f86c814fd70 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.829+0000 7f86affff640 1 -- 172.21.15.92:0/1523664102 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b800d970 con 0x7f86c814fd70 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.829+0000 7f86ce0eb640 1 -- 172.21.15.92:0/1523664102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c80c8270 con 0x7f86c814fd70 2024-04-22T07:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.829+0000 7f86ce0eb640 1 -- 172.21.15.92:0/1523664102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c80c8e90 con 0x7f86c814fd70 2024-04-22T07:04:40.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.830+0000 7f86affff640 1 -- 172.21.15.92:0/1523664102 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86b800db10 con 0x7f86c814fd70 2024-04-22T07:04:40.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.830+0000 7f86affff640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f86b406c8f0 0x7f86b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.830+0000 7f86affff640 1 -- 172.21.15.92:0/1523664102 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f86b808bb90 con 0x7f86c814fd70 2024-04-22T07:04:40.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.830+0000 7f86c6ffd640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f86b406c8f0 0x7f86b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.831+0000 7f86c6ffd640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f86b406c8f0 0x7f86b406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f86bc004840 tx=0x7f86bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:40.835+0000 7f86affff640 1 -- 172.21.15.92:0/1523664102 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86c80c8e90 con 0x7f86c814fd70 2024-04-22T07:04:41.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.010+0000 7f86ce0eb640 1 -- 172.21.15.92:0/1523664102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f86c8068b00 con 0x7f86c814fd70 2024-04-22T07:04:41.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.010+0000 7f86affff640 1 -- 172.21.15.92:0/1523664102 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f86b801a310 con 0x7f86c814fd70 2024-04-22T07:04:41.008 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-04-22T07:04:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.012+0000 7f86ce0eb640 1 -- 172.21.15.92:0/1523664102 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f86b406c8f0 msgr2=0x7f86b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.012+0000 7f86ce0eb640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f86b406c8f0 0x7f86b406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f86bc004840 tx=0x7f86bc009040 comp rx=0 tx=0).stop 2024-04-22T07:04:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.012+0000 7f86ce0eb640 1 -- 172.21.15.92:0/1523664102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c814fd70 msgr2=0x7f86c8150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.012+0000 7f86ce0eb640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c814fd70 0x7f86c8150150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f86b800f820 tx=0x7f86b800fce0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.012+0000 7f86ce0eb640 1 -- 172.21.15.92:0/1523664102 shutdown_connections 2024-04-22T07:04:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.012+0000 7f86ce0eb640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f86b406c8f0 0x7f86b406edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.012+0000 7f86ce0eb640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86c80c5f10 0x7f86c81e3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.012+0000 7f86ce0eb640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86c80c7720 0x7f86c80c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.012+0000 7f86ce0eb640 1 --2- 172.21.15.92:0/1523664102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c814fd70 0x7f86c8150150 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.012+0000 7f86ce0eb640 1 -- 172.21.15.92:0/1523664102 >> 172.21.15.92:0/1523664102 conn(0x7f86c806db80 msgr2=0x7f86c80ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.012+0000 7f86ce0eb640 1 -- 172.21.15.92:0/1523664102 shutdown_connections 2024-04-22T07:04:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.012+0000 7f86ce0eb640 1 -- 172.21.15.92:0/1523664102 wait complete. 2024-04-22T07:04:41.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:04:41.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.124+0000 7f73d4915640 1 Processor -- start 2024-04-22T07:04:41.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.124+0000 7f73d4915640 1 -- start start 2024-04-22T07:04:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73d4915640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73d0152a60 0x7f73d006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73d4915640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f73d0152e40 0x7f73d006ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73d4915640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73d006fe60 0x7f73d015bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73d4915640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f73d00702d0 con 0x7f73d006fe60 2024-04-22T07:04:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73d4915640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f73d00c7ee0 con 0x7f73d0152e40 2024-04-22T07:04:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73d4915640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f73d00c81c0 con 0x7f73d0152a60 2024-04-22T07:04:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73ced9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73d006fe60 0x7f73d015bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73ced9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73d006fe60 0x7f73d015bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51478/0 (socket says 172.21.15.92:51478) 2024-04-22T07:04:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73ced9d640 1 -- 172.21.15.92:0/3890346383 learned_addr learned my addr 172.21.15.92:0/3890346383 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73ced9d640 1 -- 172.21.15.92:0/3890346383 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73d0152a60 msgr2=0x7f73d006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73ce59c640 1 --2- 172.21.15.92:0/3890346383 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73d0152a60 0x7f73d006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73c7fff640 1 --2- 172.21.15.92:0/3890346383 >> v2:172.21.15.145:3300/0 conn(0x7f73d0152e40 0x7f73d006ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73ced9d640 1 --2- 172.21.15.92:0/3890346383 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73d0152a60 0x7f73d006c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73ced9d640 1 -- 172.21.15.92:0/3890346383 >> v2:172.21.15.145:3300/0 conn(0x7f73d0152e40 msgr2=0x7f73d006ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:41.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73ced9d640 1 --2- 172.21.15.92:0/3890346383 >> v2:172.21.15.145:3300/0 conn(0x7f73d0152e40 0x7f73d006ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.125+0000 7f73ced9d640 1 -- 172.21.15.92:0/3890346383 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d0070c30 con 0x7f73d006fe60 2024-04-22T07:04:41.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73ced9d640 1 --2- 172.21.15.92:0/3890346383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d006fe60 0x7f73d015bf50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73d0068750 tx=0x7f73c802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6992881c8003a246 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73cdd9b640 1 -- 172.21.15.92:0/3890346383 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c802ca70 con 0x7f73d006fe60 2024-04-22T07:04:41.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73cdd9b640 1 -- 172.21.15.92:0/3890346383 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c802cc10 con 0x7f73d006fe60 2024-04-22T07:04:41.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73cdd9b640 1 -- 172.21.15.92:0/3890346383 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c80343f0 con 0x7f73d006fe60 2024-04-22T07:04:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73d4915640 1 -- 172.21.15.92:0/3890346383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d006fe60 msgr2=0x7f73d015bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73d4915640 1 --2- 172.21.15.92:0/3890346383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d006fe60 0x7f73d015bf50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73d0068750 tx=0x7f73c802bed0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73d4915640 1 -- 172.21.15.92:0/3890346383 shutdown_connections 2024-04-22T07:04:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73d4915640 1 --2- 172.21.15.92:0/3890346383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d006fe60 0x7f73d015bf50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73d4915640 1 --2- 172.21.15.92:0/3890346383 >> v2:172.21.15.145:3300/0 conn(0x7f73d0152e40 0x7f73d006ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73d4915640 1 --2- 172.21.15.92:0/3890346383 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73d0152a60 0x7f73d006c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73d4915640 1 -- 172.21.15.92:0/3890346383 >> 172.21.15.92:0/3890346383 conn(0x7f73d00cd440 msgr2=0x7f73d00cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73d4915640 1 -- 172.21.15.92:0/3890346383 shutdown_connections 2024-04-22T07:04:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.126+0000 7f73d4915640 1 -- 172.21.15.92:0/3890346383 wait complete. 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.128+0000 7f73d4915640 1 Processor -- start 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.128+0000 7f73d4915640 1 -- start start 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.128+0000 7f73d4915640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d006fe60 0x7f73d01ef480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.128+0000 7f73d4915640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73d0152a60 0x7f73d0149af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.128+0000 7f73d4915640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0152e40 0x7f73d01e4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.128+0000 7f73d4915640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f73d00c7560 con 0x7f73d006fe60 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.128+0000 7f73d4915640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0070570 con 0x7f73d0152e40 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.128+0000 7f73d4915640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f73d00c76e0 con 0x7f73d0152a60 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.128+0000 7f73ce59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d006fe60 0x7f73d01ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73ce59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d006fe60 0x7f73d01ef480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51492/0 (socket says 172.21.15.92:51492) 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73ce59c640 1 -- 172.21.15.92:0/4005784129 learned_addr learned my addr 172.21.15.92:0/4005784129 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73c7fff640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73d0152a60 0x7f73d0149af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73ced9d640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0152e40 0x7f73d01e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73ce59c640 1 -- 172.21.15.92:0/4005784129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73d0152a60 msgr2=0x7f73d0149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73ce59c640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73d0152a60 0x7f73d0149af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73ce59c640 1 -- 172.21.15.92:0/4005784129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0152e40 msgr2=0x7f73d01e4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73ce59c640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0152e40 0x7f73d01e4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73ce59c640 1 -- 172.21.15.92:0/4005784129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d014a030 con 0x7f73d006fe60 2024-04-22T07:04:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73ce59c640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d006fe60 0x7f73d01ef480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f73c000c7f0 tx=0x7f73c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73c6ffd640 1 -- 172.21.15.92:0/4005784129 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0015070 con 0x7f73d006fe60 2024-04-22T07:04:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73c6ffd640 1 -- 172.21.15.92:0/4005784129 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c0004240 con 0x7f73d006fe60 2024-04-22T07:04:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73d4915640 1 -- 172.21.15.92:0/4005784129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d0149050 con 0x7f73d006fe60 2024-04-22T07:04:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73c6ffd640 1 -- 172.21.15.92:0/4005784129 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0007620 con 0x7f73d006fe60 2024-04-22T07:04:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.129+0000 7f73d4915640 1 -- 172.21.15.92:0/4005784129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d014bbe0 con 0x7f73d006fe60 2024-04-22T07:04:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.130+0000 7f73d4915640 1 -- 172.21.15.92:0/4005784129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d0069420 con 0x7f73d006fe60 2024-04-22T07:04:41.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.131+0000 7f73c6ffd640 1 -- 172.21.15.92:0/4005784129 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73c00077c0 con 0x7f73d006fe60 2024-04-22T07:04:41.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.131+0000 7f73c6ffd640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f73b006c8f0 0x7f73b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.131+0000 7f73c6ffd640 1 -- 172.21.15.92:0/4005784129 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f73c008ba30 con 0x7f73d006fe60 2024-04-22T07:04:41.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.131+0000 7f73c7fff640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f73b006c8f0 0x7f73b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.132+0000 7f73c7fff640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f73b006c8f0 0x7f73b006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f73d01510b0 tx=0x7f73b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.136+0000 7f73c6ffd640 1 -- 172.21.15.92:0/4005784129 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f73d0069420 con 0x7f73d006fe60 2024-04-22T07:04:41.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.308+0000 7f73d4915640 1 -- 172.21.15.92:0/4005784129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f73d0069930 con 0x7f73d006fe60 2024-04-22T07:04:41.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.308+0000 7f73c6ffd640 1 -- 172.21.15.92:0/4005784129 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f73c0059e60 con 0x7f73d006fe60 2024-04-22T07:04:41.307 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-04-22T07:04:41.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.310+0000 7f73d4915640 1 -- 172.21.15.92:0/4005784129 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f73b006c8f0 msgr2=0x7f73b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.310+0000 7f73d4915640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f73b006c8f0 0x7f73b006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f73d01510b0 tx=0x7f73b8008040 comp rx=0 tx=0).stop 2024-04-22T07:04:41.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.310+0000 7f73d4915640 1 -- 172.21.15.92:0/4005784129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d006fe60 msgr2=0x7f73d01ef480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.310+0000 7f73d4915640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d006fe60 0x7f73d01ef480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f73c000c7f0 tx=0x7f73c000ccb0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.310+0000 7f73d4915640 1 -- 172.21.15.92:0/4005784129 shutdown_connections 2024-04-22T07:04:41.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.310+0000 7f73d4915640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f73b006c8f0 0x7f73b006edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.310+0000 7f73d4915640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0152e40 0x7f73d01e4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.310+0000 7f73d4915640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73d0152a60 0x7f73d0149af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.310+0000 7f73d4915640 1 --2- 172.21.15.92:0/4005784129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73d006fe60 0x7f73d01ef480 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.310+0000 7f73d4915640 1 -- 172.21.15.92:0/4005784129 >> 172.21.15.92:0/4005784129 conn(0x7f73d00cd440 msgr2=0x7f73d014d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.310+0000 7f73d4915640 1 -- 172.21.15.92:0/4005784129 shutdown_connections 2024-04-22T07:04:41.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.310+0000 7f73d4915640 1 -- 172.21.15.92:0/4005784129 wait complete. 2024-04-22T07:04:41.319 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-22T07:04:41.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.417+0000 7f5a53481640 1 Processor -- start 2024-04-22T07:04:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.418+0000 7f5a53481640 1 -- start start 2024-04-22T07:04:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.418+0000 7f5a53481640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a4c15ee60 0x7f5a4c0c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.418+0000 7f5a53481640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a4c068230 con 0x7f5a4c06b970 2024-04-22T07:04:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.418+0000 7f5a53481640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a4c0684d0 con 0x7f5a4c154f40 2024-04-22T07:04:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.418+0000 7f5a53481640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f5a4c06cae0 con 0x7f5a4c15ee60 2024-04-22T07:04:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.419+0000 7f5a51a1e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a4c15ee60 0x7f5a4c0c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.419+0000 7f5a50a1c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5a4c06b970 0x7f5a4c15e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48310/0 (socket says 172.21.15.92:48310) 2024-04-22T07:04:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.419+0000 7f5a50a1c640 1 -- 172.21.15.92:0/1492428675 learned_addr learned my addr 172.21.15.92:0/1492428675 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.419+0000 7f5a51a1e640 1 -- 172.21.15.92:0/1492428675 >> v1:172.21.15.145:6789/0 conn(0x7f5a4c154f40 legacy=0x7f5a4c0cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.419+0000 7f5a51a1e640 1 -- 172.21.15.92:0/1492428675 >> v1:172.21.15.92:6789/0 conn(0x7f5a4c06b970 legacy=0x7f5a4c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.419+0000 7f5a43fff640 1 -- 172.21.15.92:0/1492428675 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2256523703 0 0) 0x7f5a4c068230 con 0x7f5a4c06b970 2024-04-22T07:04:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.419+0000 7f5a51a1e640 1 -- 172.21.15.92:0/1492428675 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c06cdc0 con 0x7f5a4c15ee60 2024-04-22T07:04:41.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.420+0000 7f5a51a1e640 1 --2- 172.21.15.92:0/1492428675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c15ee60 0x7f5a4c0c9160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0686a0 tx=0x7f5a4402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49936167af9dc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.420+0000 7f5a43fff640 1 -- 172.21.15.92:0/1492428675 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44039070 con 0x7f5a4c15ee60 2024-04-22T07:04:41.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.420+0000 7f5a43fff640 1 -- 172.21.15.92:0/1492428675 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a4402cd80 con 0x7f5a4c15ee60 2024-04-22T07:04:41.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.420+0000 7f5a43fff640 1 -- 172.21.15.92:0/1492428675 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44034660 con 0x7f5a4c15ee60 2024-04-22T07:04:41.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.420+0000 7f5a53481640 1 -- 172.21.15.92:0/1492428675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c15ee60 msgr2=0x7f5a4c0c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.420+0000 7f5a53481640 1 --2- 172.21.15.92:0/1492428675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c15ee60 0x7f5a4c0c9160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0686a0 tx=0x7f5a4402bfd0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.420+0000 7f5a53481640 1 -- 172.21.15.92:0/1492428675 shutdown_connections 2024-04-22T07:04:41.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.420+0000 7f5a53481640 1 --2- 172.21.15.92:0/1492428675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c15ee60 0x7f5a4c0c9160 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.420+0000 7f5a53481640 1 -- 172.21.15.92:0/1492428675 >> 172.21.15.92:0/1492428675 conn(0x7f5a4c072ee0 msgr2=0x7f5a4c06e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.420+0000 7f5a53481640 1 -- 172.21.15.92:0/1492428675 shutdown_connections 2024-04-22T07:04:41.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.420+0000 7f5a53481640 1 -- 172.21.15.92:0/1492428675 wait complete. 2024-04-22T07:04:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.421+0000 7f5a53481640 1 Processor -- start 2024-04-22T07:04:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.422+0000 7f5a53481640 1 -- start start 2024-04-22T07:04:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.422+0000 7f5a53481640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a4c06b970 0x7f5a4c147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.422+0000 7f5a53481640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c154f40 0x7f5a4c146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.422+0000 7f5a53481640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c15ee60 0x7f5a4c0c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.422+0000 7f5a53481640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1554a0 con 0x7f5a4c15ee60 2024-04-22T07:04:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.422+0000 7f5a53481640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c155320 con 0x7f5a4c154f40 2024-04-22T07:04:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.422+0000 7f5a53481640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4c0cd9e0 con 0x7f5a4c06b970 2024-04-22T07:04:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.422+0000 7f5a51a1e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c15ee60 0x7f5a4c0c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.422+0000 7f5a51a1e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c15ee60 0x7f5a4c0c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51502/0 (socket says 172.21.15.92:51502) 2024-04-22T07:04:41.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.422+0000 7f5a51a1e640 1 -- 172.21.15.92:0/218474680 learned_addr learned my addr 172.21.15.92:0/218474680 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.422+0000 7f5a5121d640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a4c06b970 0x7f5a4c147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a50a1c640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c154f40 0x7f5a4c146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a51a1e640 1 -- 172.21.15.92:0/218474680 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a4c06b970 msgr2=0x7f5a4c147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a51a1e640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a4c06b970 0x7f5a4c147b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a51a1e640 1 -- 172.21.15.92:0/218474680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c154f40 msgr2=0x7f5a4c146f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a51a1e640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c154f40 0x7f5a4c146f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a51a1e640 1 -- 172.21.15.92:0/218474680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c1e6bf0 con 0x7f5a4c15ee60 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a50a1c640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c154f40 0x7f5a4c146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a51a1e640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c15ee60 0x7f5a4c0c6190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009e80 tx=0x7f5a44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a41ffb640 1 -- 172.21.15.92:0/218474680 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44044020 con 0x7f5a4c15ee60 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a41ffb640 1 -- 172.21.15.92:0/218474680 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a4403a040 con 0x7f5a4c15ee60 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a53481640 1 -- 172.21.15.92:0/218474680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c148060 con 0x7f5a4c15ee60 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a41ffb640 1 -- 172.21.15.92:0/218474680 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44039050 con 0x7f5a4c15ee60 2024-04-22T07:04:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.423+0000 7f5a53481640 1 -- 172.21.15.92:0/218474680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c1e67e0 con 0x7f5a4c15ee60 2024-04-22T07:04:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.424+0000 7f5a53481640 1 -- 172.21.15.92:0/218474680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a4c068b00 con 0x7f5a4c15ee60 2024-04-22T07:04:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.425+0000 7f5a41ffb640 1 -- 172.21.15.92:0/218474680 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a44003b90 con 0x7f5a4c15ee60 2024-04-22T07:04:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.425+0000 7f5a41ffb640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5a3006c920 0x7f5a3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.425+0000 7f5a5121d640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5a3006c920 0x7f5a3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.425+0000 7f5a41ffb640 1 -- 172.21.15.92:0/218474680 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5a440ae350 con 0x7f5a4c15ee60 2024-04-22T07:04:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.426+0000 7f5a5121d640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5a3006c920 0x7f5a3006ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000f60 tx=0x7f5a3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.430+0000 7f5a41ffb640 1 -- 172.21.15.92:0/218474680 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a4404e020 con 0x7f5a4c15ee60 2024-04-22T07:04:41.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.599+0000 7f5a53481640 1 -- 172.21.15.92:0/218474680 --> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a4c06c8c0 con 0x7f5a3006c920 2024-04-22T07:04:41.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.605+0000 7f5a41ffb640 1 -- 172.21.15.92:0/218474680 <== mgr.4107 v2:172.21.15.92:6832/479317737 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251928 (secure 0 0 0) 0x7f5a4c06c8c0 con 0x7f5a3006c920 2024-04-22T07:04:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:41.607 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-22T07:04:41.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.608+0000 7f5a53481640 1 -- 172.21.15.92:0/218474680 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5a3006c920 msgr2=0x7f5a3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.608+0000 7f5a53481640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5a3006c920 0x7f5a3006ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000f60 tx=0x7f5a3c009040 comp rx=0 tx=0).stop 2024-04-22T07:04:41.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.608+0000 7f5a53481640 1 -- 172.21.15.92:0/218474680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c15ee60 msgr2=0x7f5a4c0c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.608+0000 7f5a53481640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c15ee60 0x7f5a4c0c6190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009e80 tx=0x7f5a44004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:41.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.608+0000 7f5a53481640 1 -- 172.21.15.92:0/218474680 shutdown_connections 2024-04-22T07:04:41.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.608+0000 7f5a53481640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f5a3006c920 0x7f5a3006ede0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.608+0000 7f5a53481640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c15ee60 0x7f5a4c0c6190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.608+0000 7f5a53481640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c154f40 0x7f5a4c146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.608+0000 7f5a53481640 1 --2- 172.21.15.92:0/218474680 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a4c06b970 0x7f5a4c147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.608+0000 7f5a53481640 1 -- 172.21.15.92:0/218474680 >> 172.21.15.92:0/218474680 conn(0x7f5a4c072ee0 msgr2=0x7f5a4c0c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.609+0000 7f5a53481640 1 -- 172.21.15.92:0/218474680 shutdown_connections 2024-04-22T07:04:41.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.609+0000 7f5a53481640 1 -- 172.21.15.92:0/218474680 wait complete. 2024-04-22T07:04:41.617 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-22T07:04:39.939956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":364,"num_write_kb":2478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":3088384,"data_stored":1764276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002350"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862171+0000","last_change":"2024-04-22T07:04:03.796924+0000","last_active":"2024-04-22T07:04:04.862171+0000","last_peered":"2024-04-22T07:04:04.862171+0000","last_clean":"2024-04-22T07:04:04.862171+0000","last_became_active":"2024-04-22T07:04:03.796652+0000","last_became_peered":"2024-04-22T07:04:03.796652+0000","last_unstale":"2024-04-22T07:04:04.862171+0000","last_undegraded":"2024-04-22T07:04:04.862171+0000","last_fullsized":"2024-04-22T07:04:04.862171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:01.867906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788117+0000","last_change":"2024-04-22T07:04:03.793652+0000","last_active":"2024-04-22T07:04:04.788117+0000","last_peered":"2024-04-22T07:04:04.788117+0000","last_clean":"2024-04-22T07:04:04.788117+0000","last_became_active":"2024-04-22T07:04:03.793397+0000","last_became_peered":"2024-04-22T07:04:03.793397+0000","last_unstale":"2024-04-22T07:04:04.788117+0000","last_undegraded":"2024-04-22T07:04:04.788117+0000","last_fullsized":"2024-04-22T07:04:04.788117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859908+0000","last_change":"2024-04-22T07:04:03.788534+0000","last_active":"2024-04-22T07:04:04.859908+0000","last_peered":"2024-04-22T07:04:04.859908+0000","last_clean":"2024-04-22T07:04:04.859908+0000","last_became_active":"2024-04-22T07:04:03.787747+0000","last_became_peered":"2024-04-22T07:04:03.787747+0000","last_unstale":"2024-04-22T07:04:04.859908+0000","last_undegraded":"2024-04-22T07:04:04.859908+0000","last_fullsized":"2024-04-22T07:04:04.859908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861337+0000","last_change":"2024-04-22T07:04:03.800150+0000","last_active":"2024-04-22T07:04:04.861337+0000","last_peered":"2024-04-22T07:04:04.861337+0000","last_clean":"2024-04-22T07:04:04.861337+0000","last_became_active":"2024-04-22T07:04:03.799887+0000","last_became_peered":"2024-04-22T07:04:03.799887+0000","last_unstale":"2024-04-22T07:04:04.861337+0000","last_undegraded":"2024-04-22T07:04:04.861337+0000","last_fullsized":"2024-04-22T07:04:04.861337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861727+0000","last_change":"2024-04-22T07:04:03.798482+0000","last_active":"2024-04-22T07:04:04.861727+0000","last_peered":"2024-04-22T07:04:04.861727+0000","last_clean":"2024-04-22T07:04:04.861727+0000","last_became_active":"2024-04-22T07:04:03.798205+0000","last_became_peered":"2024-04-22T07:04:03.798205+0000","last_unstale":"2024-04-22T07:04:04.861727+0000","last_undegraded":"2024-04-22T07:04:04.861727+0000","last_fullsized":"2024-04-22T07:04:04.861727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861044+0000","last_change":"2024-04-22T07:04:03.797373+0000","last_active":"2024-04-22T07:04:04.861044+0000","last_peered":"2024-04-22T07:04:04.861044+0000","last_clean":"2024-04-22T07:04:04.861044+0000","last_became_active":"2024-04-22T07:04:03.797090+0000","last_became_peered":"2024-04-22T07:04:03.797090+0000","last_unstale":"2024-04-22T07:04:04.861044+0000","last_undegraded":"2024-04-22T07:04:04.861044+0000","last_fullsized":"2024-04-22T07:04:04.861044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861437+0000","last_change":"2024-04-22T07:04:03.793725+0000","last_active":"2024-04-22T07:04:04.861437+0000","last_peered":"2024-04-22T07:04:04.861437+0000","last_clean":"2024-04-22T07:04:04.861437+0000","last_became_active":"2024-04-22T07:04:03.793418+0000","last_became_peered":"2024-04-22T07:04:03.793418+0000","last_unstale":"2024-04-22T07:04:04.861437+0000","last_undegraded":"2024-04-22T07:04:04.861437+0000","last_fullsized":"2024-04-22T07:04:04.861437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859500+0000","last_change":"2024-04-22T07:04:03.795377+0000","last_active":"2024-04-22T07:04:04.859500+0000","last_peered":"2024-04-22T07:04:04.859500+0000","last_clean":"2024-04-22T07:04:04.859500+0000","last_became_active":"2024-04-22T07:04:03.795106+0000","last_became_peered":"2024-04-22T07:04:03.795106+0000","last_unstale":"2024-04-22T07:04:04.859500+0000","last_undegraded":"2024-04-22T07:04:04.859500+0000","last_fullsized":"2024-04-22T07:04:04.859500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862209+0000","last_change":"2024-04-22T07:04:03.797692+0000","last_active":"2024-04-22T07:04:04.862209+0000","last_peered":"2024-04-22T07:04:04.862209+0000","last_clean":"2024-04-22T07:04:04.862209+0000","last_became_active":"2024-04-22T07:04:03.797396+0000","last_became_peered":"2024-04-22T07:04:03.797396+0000","last_unstale":"2024-04-22T07:04:04.862209+0000","last_undegraded":"2024-04-22T07:04:04.862209+0000","last_fullsized":"2024-04-22T07:04:04.862209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:39.068857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860742+0000","last_change":"2024-04-22T07:04:03.797228+0000","last_active":"2024-04-22T07:04:04.860742+0000","last_peered":"2024-04-22T07:04:04.860742+0000","last_clean":"2024-04-22T07:04:04.860742+0000","last_became_active":"2024-04-22T07:04:03.796614+0000","last_became_peered":"2024-04-22T07:04:03.796614+0000","last_unstale":"2024-04-22T07:04:04.860742+0000","last_undegraded":"2024-04-22T07:04:04.860742+0000","last_fullsized":"2024-04-22T07:04:04.860742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860497+0000","last_change":"2024-04-22T07:04:03.799434+0000","last_active":"2024-04-22T07:04:04.860497+0000","last_peered":"2024-04-22T07:04:04.860497+0000","last_clean":"2024-04-22T07:04:04.860497+0000","last_became_active":"2024-04-22T07:04:03.799173+0000","last_became_peered":"2024-04-22T07:04:03.799173+0000","last_unstale":"2024-04-22T07:04:04.860497+0000","last_undegraded":"2024-04-22T07:04:04.860497+0000","last_fullsized":"2024-04-22T07:04:04.860497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790488+0000","last_change":"2024-04-22T07:04:03.800035+0000","last_active":"2024-04-22T07:04:04.790488+0000","last_peered":"2024-04-22T07:04:04.790488+0000","last_clean":"2024-04-22T07:04:04.790488+0000","last_became_active":"2024-04-22T07:04:03.799646+0000","last_became_peered":"2024-04-22T07:04:03.799646+0000","last_unstale":"2024-04-22T07:04:04.790488+0000","last_undegraded":"2024-04-22T07:04:04.790488+0000","last_fullsized":"2024-04-22T07:04:04.790488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:17.287068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860651+0000","last_change":"2024-04-22T07:04:03.797324+0000","last_active":"2024-04-22T07:04:04.860651+0000","last_peered":"2024-04-22T07:04:04.860651+0000","last_clean":"2024-04-22T07:04:04.860651+0000","last_became_active":"2024-04-22T07:04:03.797039+0000","last_became_peered":"2024-04-22T07:04:03.797039+0000","last_unstale":"2024-04-22T07:04:04.860651+0000","last_undegraded":"2024-04-22T07:04:04.860651+0000","last_fullsized":"2024-04-22T07:04:04.860651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859720+0000","last_change":"2024-04-22T07:04:03.790732+0000","last_active":"2024-04-22T07:04:04.859720+0000","last_peered":"2024-04-22T07:04:04.859720+0000","last_clean":"2024-04-22T07:04:04.859720+0000","last_became_active":"2024-04-22T07:04:03.788340+0000","last_became_peered":"2024-04-22T07:04:03.788340+0000","last_unstale":"2024-04-22T07:04:04.859720+0000","last_undegraded":"2024-04-22T07:04:04.859720+0000","last_fullsized":"2024-04-22T07:04:04.859720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787860+0000","last_change":"2024-04-22T07:04:03.799368+0000","last_active":"2024-04-22T07:04:04.787860+0000","last_peered":"2024-04-22T07:04:04.787860+0000","last_clean":"2024-04-22T07:04:04.787860+0000","last_became_active":"2024-04-22T07:04:03.799067+0000","last_became_peered":"2024-04-22T07:04:03.799067+0000","last_unstale":"2024-04-22T07:04:04.787860+0000","last_undegraded":"2024-04-22T07:04:04.787860+0000","last_fullsized":"2024-04-22T07:04:04.787860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860988+0000","last_change":"2024-04-22T07:04:03.802947+0000","last_active":"2024-04-22T07:04:04.860988+0000","last_peered":"2024-04-22T07:04:04.860988+0000","last_clean":"2024-04-22T07:04:04.860988+0000","last_became_active":"2024-04-22T07:04:03.802493+0000","last_became_peered":"2024-04-22T07:04:03.802493+0000","last_unstale":"2024-04-22T07:04:04.860988+0000","last_undegraded":"2024-04-22T07:04:04.860988+0000","last_fullsized":"2024-04-22T07:04:04.860988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860714+0000","last_change":"2024-04-22T07:04:03.795581+0000","last_active":"2024-04-22T07:04:04.860714+0000","last_peered":"2024-04-22T07:04:04.860714+0000","last_clean":"2024-04-22T07:04:04.860714+0000","last_became_active":"2024-04-22T07:04:03.795292+0000","last_became_peered":"2024-04-22T07:04:03.795292+0000","last_unstale":"2024-04-22T07:04:04.860714+0000","last_undegraded":"2024-04-22T07:04:04.860714+0000","last_fullsized":"2024-04-22T07:04:04.860714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859993+0000","last_change":"2024-04-22T07:04:03.796132+0000","last_active":"2024-04-22T07:04:04.859993+0000","last_peered":"2024-04-22T07:04:04.859993+0000","last_clean":"2024-04-22T07:04:04.859993+0000","last_became_active":"2024-04-22T07:04:03.795069+0000","last_became_peered":"2024-04-22T07:04:03.795069+0000","last_unstale":"2024-04-22T07:04:04.859993+0000","last_undegraded":"2024-04-22T07:04:04.859993+0000","last_fullsized":"2024-04-22T07:04:04.859993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:08.934970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862125+0000","last_change":"2024-04-22T07:04:03.794552+0000","last_active":"2024-04-22T07:04:04.862125+0000","last_peered":"2024-04-22T07:04:04.862125+0000","last_clean":"2024-04-22T07:04:04.862125+0000","last_became_active":"2024-04-22T07:04:03.794046+0000","last_became_peered":"2024-04-22T07:04:03.794046+0000","last_unstale":"2024-04-22T07:04:04.862125+0000","last_undegraded":"2024-04-22T07:04:04.862125+0000","last_fullsized":"2024-04-22T07:04:04.862125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860703+0000","last_change":"2024-04-22T07:04:03.798686+0000","last_active":"2024-04-22T07:04:04.860703+0000","last_peered":"2024-04-22T07:04:04.860703+0000","last_clean":"2024-04-22T07:04:04.860703+0000","last_became_active":"2024-04-22T07:04:03.798104+0000","last_became_peered":"2024-04-22T07:04:03.798104+0000","last_unstale":"2024-04-22T07:04:04.860703+0000","last_undegraded":"2024-04-22T07:04:04.860703+0000","last_fullsized":"2024-04-22T07:04:04.860703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860494+0000","last_change":"2024-04-22T07:04:03.794215+0000","last_active":"2024-04-22T07:04:04.860494+0000","last_peered":"2024-04-22T07:04:04.860494+0000","last_clean":"2024-04-22T07:04:04.860494+0000","last_became_active":"2024-04-22T07:04:03.793990+0000","last_became_peered":"2024-04-22T07:04:03.793990+0000","last_unstale":"2024-04-22T07:04:04.860494+0000","last_undegraded":"2024-04-22T07:04:04.860494+0000","last_fullsized":"2024-04-22T07:04:04.860494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861162+0000","last_change":"2024-04-22T07:04:03.798539+0000","last_active":"2024-04-22T07:04:04.861162+0000","last_peered":"2024-04-22T07:04:04.861162+0000","last_clean":"2024-04-22T07:04:04.861162+0000","last_became_active":"2024-04-22T07:04:03.797713+0000","last_became_peered":"2024-04-22T07:04:03.797713+0000","last_unstale":"2024-04-22T07:04:04.861162+0000","last_undegraded":"2024-04-22T07:04:04.861162+0000","last_fullsized":"2024-04-22T07:04:04.861162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859945+0000","last_change":"2024-04-22T07:04:03.798610+0000","last_active":"2024-04-22T07:04:04.859945+0000","last_peered":"2024-04-22T07:04:04.859945+0000","last_clean":"2024-04-22T07:04:04.859945+0000","last_became_active":"2024-04-22T07:04:03.797981+0000","last_became_peered":"2024-04-22T07:04:03.797981+0000","last_unstale":"2024-04-22T07:04:04.859945+0000","last_undegraded":"2024-04-22T07:04:04.859945+0000","last_fullsized":"2024-04-22T07:04:04.859945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861507+0000","last_change":"2024-04-22T07:04:03.802742+0000","last_active":"2024-04-22T07:04:04.861507+0000","last_peered":"2024-04-22T07:04:04.861507+0000","last_clean":"2024-04-22T07:04:04.861507+0000","last_became_active":"2024-04-22T07:04:03.802127+0000","last_became_peered":"2024-04-22T07:04:03.802127+0000","last_unstale":"2024-04-22T07:04:04.861507+0000","last_undegraded":"2024-04-22T07:04:04.861507+0000","last_fullsized":"2024-04-22T07:04:04.861507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860138+0000","last_change":"2024-04-22T07:04:03.798789+0000","last_active":"2024-04-22T07:04:04.860138+0000","last_peered":"2024-04-22T07:04:04.860138+0000","last_clean":"2024-04-22T07:04:04.860138+0000","last_became_active":"2024-04-22T07:04:03.798505+0000","last_became_peered":"2024-04-22T07:04:03.798505+0000","last_unstale":"2024-04-22T07:04:04.860138+0000","last_undegraded":"2024-04-22T07:04:04.860138+0000","last_fullsized":"2024-04-22T07:04:04.860138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860919+0000","last_change":"2024-04-22T07:04:03.794111+0000","last_active":"2024-04-22T07:04:04.860919+0000","last_peered":"2024-04-22T07:04:04.860919+0000","last_clean":"2024-04-22T07:04:04.860919+0000","last_became_active":"2024-04-22T07:04:03.793682+0000","last_became_peered":"2024-04-22T07:04:03.793682+0000","last_unstale":"2024-04-22T07:04:04.860919+0000","last_undegraded":"2024-04-22T07:04:04.860919+0000","last_fullsized":"2024-04-22T07:04:04.860919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788083+0000","last_change":"2024-04-22T07:04:03.790167+0000","last_active":"2024-04-22T07:04:04.788083+0000","last_peered":"2024-04-22T07:04:04.788083+0000","last_clean":"2024-04-22T07:04:04.788083+0000","last_became_active":"2024-04-22T07:04:03.789873+0000","last_became_peered":"2024-04-22T07:04:03.789873+0000","last_unstale":"2024-04-22T07:04:04.788083+0000","last_undegraded":"2024-04-22T07:04:04.788083+0000","last_fullsized":"2024-04-22T07:04:04.788083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859954+0000","last_change":"2024-04-22T07:04:03.798713+0000","last_active":"2024-04-22T07:04:04.859954+0000","last_peered":"2024-04-22T07:04:04.859954+0000","last_clean":"2024-04-22T07:04:04.859954+0000","last_became_active":"2024-04-22T07:04:03.798344+0000","last_became_peered":"2024-04-22T07:04:03.798344+0000","last_unstale":"2024-04-22T07:04:04.859954+0000","last_undegraded":"2024-04-22T07:04:04.859954+0000","last_fullsized":"2024-04-22T07:04:04.859954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860634+0000","last_change":"2024-04-22T07:04:03.795452+0000","last_active":"2024-04-22T07:04:04.860634+0000","last_peered":"2024-04-22T07:04:04.860634+0000","last_clean":"2024-04-22T07:04:04.860634+0000","last_became_active":"2024-04-22T07:04:03.794970+0000","last_became_peered":"2024-04-22T07:04:03.794970+0000","last_unstale":"2024-04-22T07:04:04.860634+0000","last_undegraded":"2024-04-22T07:04:04.860634+0000","last_fullsized":"2024-04-22T07:04:04.860634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861203+0000","last_change":"2024-04-22T07:04:03.799581+0000","last_active":"2024-04-22T07:04:04.861203+0000","last_peered":"2024-04-22T07:04:04.861203+0000","last_clean":"2024-04-22T07:04:04.861203+0000","last_became_active":"2024-04-22T07:04:03.799328+0000","last_became_peered":"2024-04-22T07:04:03.799328+0000","last_unstale":"2024-04-22T07:04:04.861203+0000","last_undegraded":"2024-04-22T07:04:04.861203+0000","last_fullsized":"2024-04-22T07:04:04.861203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859944+0000","last_change":"2024-04-22T07:04:03.796830+0000","last_active":"2024-04-22T07:04:04.859944+0000","last_peered":"2024-04-22T07:04:04.859944+0000","last_clean":"2024-04-22T07:04:04.859944+0000","last_became_active":"2024-04-22T07:04:03.796551+0000","last_became_peered":"2024-04-22T07:04:03.796551+0000","last_unstale":"2024-04-22T07:04:04.859944+0000","last_undegraded":"2024-04-22T07:04:04.859944+0000","last_fullsized":"2024-04-22T07:04:04.859944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860101+0000","last_change":"2024-04-22T07:04:03.795320+0000","last_active":"2024-04-22T07:04:04.860101+0000","last_peered":"2024-04-22T07:04:04.860101+0000","last_clean":"2024-04-22T07:04:04.860101+0000","last_became_active":"2024-04-22T07:04:03.795025+0000","last_became_peered":"2024-04-22T07:04:03.795025+0000","last_unstale":"2024-04-22T07:04:04.860101+0000","last_undegraded":"2024-04-22T07:04:04.860101+0000","last_fullsized":"2024-04-22T07:04:04.860101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790119+0000","last_change":"2024-04-22T07:04:03.788859+0000","last_active":"2024-04-22T07:04:04.790119+0000","last_peered":"2024-04-22T07:04:04.790119+0000","last_clean":"2024-04-22T07:04:04.790119+0000","last_became_active":"2024-04-22T07:04:03.788573+0000","last_became_peered":"2024-04-22T07:04:03.788573+0000","last_unstale":"2024-04-22T07:04:04.790119+0000","last_undegraded":"2024-04-22T07:04:04.790119+0000","last_fullsized":"2024-04-22T07:04:04.790119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:36.716689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.826359+0000","last_change":"2024-04-22T07:04:02.795724+0000","last_active":"2024-04-22T07:04:13.826359+0000","last_peered":"2024-04-22T07:04:13.826359+0000","last_clean":"2024-04-22T07:04:13.826359+0000","last_became_active":"2024-04-22T07:04:02.795371+0000","last_became_peered":"2024-04-22T07:04:02.795371+0000","last_unstale":"2024-04-22T07:04:13.826359+0000","last_undegraded":"2024-04-22T07:04:13.826359+0000","last_fullsized":"2024-04-22T07:04:13.826359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860689+0000","last_change":"2024-04-22T07:04:03.787030+0000","last_active":"2024-04-22T07:04:04.860689+0000","last_peered":"2024-04-22T07:04:04.860689+0000","last_clean":"2024-04-22T07:04:04.860689+0000","last_became_active":"2024-04-22T07:04:03.786357+0000","last_became_peered":"2024-04-22T07:04:03.786357+0000","last_unstale":"2024-04-22T07:04:04.860689+0000","last_undegraded":"2024-04-22T07:04:04.860689+0000","last_fullsized":"2024-04-22T07:04:04.860689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.004655+0000","last_change":"2024-04-22T07:04:02.791429+0000","last_active":"2024-04-22T07:04:24.004655+0000","last_peered":"2024-04-22T07:04:24.004655+0000","last_clean":"2024-04-22T07:04:24.004655+0000","last_became_active":"2024-04-22T07:04:02.791114+0000","last_became_peered":"2024-04-22T07:04:02.791114+0000","last_unstale":"2024-04-22T07:04:24.004655+0000","last_undegraded":"2024-04-22T07:04:24.004655+0000","last_fullsized":"2024-04-22T07:04:24.004655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.917309+0000","last_change":"2024-04-22T07:04:02.789531+0000","last_active":"2024-04-22T07:04:18.917309+0000","last_peered":"2024-04-22T07:04:18.917309+0000","last_clean":"2024-04-22T07:04:18.917309+0000","last_became_active":"2024-04-22T07:04:02.789248+0000","last_became_peered":"2024-04-22T07:04:02.789248+0000","last_unstale":"2024-04-22T07:04:18.917309+0000","last_undegraded":"2024-04-22T07:04:18.917309+0000","last_fullsized":"2024-04-22T07:04:18.917309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:06:43.031782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859624+0000","last_change":"2024-04-22T07:04:03.790142+0000","last_active":"2024-04-22T07:04:04.859624+0000","last_peered":"2024-04-22T07:04:04.859624+0000","last_clean":"2024-04-22T07:04:04.859624+0000","last_became_active":"2024-04-22T07:04:03.789869+0000","last_became_peered":"2024-04-22T07:04:03.789869+0000","last_unstale":"2024-04-22T07:04:04.859624+0000","last_undegraded":"2024-04-22T07:04:04.859624+0000","last_fullsized":"2024-04-22T07:04:04.859624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.002089+0000","last_change":"2024-04-22T07:04:02.790537+0000","last_active":"2024-04-22T07:04:24.002089+0000","last_peered":"2024-04-22T07:04:24.002089+0000","last_clean":"2024-04-22T07:04:24.002089+0000","last_became_active":"2024-04-22T07:04:02.789072+0000","last_became_peered":"2024-04-22T07:04:02.789072+0000","last_unstale":"2024-04-22T07:04:24.002089+0000","last_undegraded":"2024-04-22T07:04:24.002089+0000","last_fullsized":"2024-04-22T07:04:24.002089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787835+0000","last_change":"2024-04-22T07:04:03.791239+0000","last_active":"2024-04-22T07:04:04.787835+0000","last_peered":"2024-04-22T07:04:04.787835+0000","last_clean":"2024-04-22T07:04:04.787835+0000","last_became_active":"2024-04-22T07:04:03.790515+0000","last_became_peered":"2024-04-22T07:04:03.790515+0000","last_unstale":"2024-04-22T07:04:04.787835+0000","last_undegraded":"2024-04-22T07:04:04.787835+0000","last_fullsized":"2024-04-22T07:04:04.787835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.825000+0000","last_change":"2024-04-22T07:04:02.793582+0000","last_active":"2024-04-22T07:04:13.825000+0000","last_peered":"2024-04-22T07:04:13.825000+0000","last_clean":"2024-04-22T07:04:13.825000+0000","last_became_active":"2024-04-22T07:04:02.793281+0000","last_became_peered":"2024-04-22T07:04:02.793281+0000","last_unstale":"2024-04-22T07:04:13.825000+0000","last_undegraded":"2024-04-22T07:04:13.825000+0000","last_fullsized":"2024-04-22T07:04:13.825000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860510+0000","last_change":"2024-04-22T07:04:03.793994+0000","last_active":"2024-04-22T07:04:04.860510+0000","last_peered":"2024-04-22T07:04:04.860510+0000","last_clean":"2024-04-22T07:04:04.860510+0000","last_became_active":"2024-04-22T07:04:03.793734+0000","last_became_peered":"2024-04-22T07:04:03.793734+0000","last_unstale":"2024-04-22T07:04:04.860510+0000","last_undegraded":"2024-04-22T07:04:04.860510+0000","last_fullsized":"2024-04-22T07:04:04.860510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.034436+0000","last_change":"2024-04-22T07:04:02.791945+0000","last_active":"2024-04-22T07:04:30.034436+0000","last_peered":"2024-04-22T07:04:30.034436+0000","last_clean":"2024-04-22T07:04:30.034436+0000","last_became_active":"2024-04-22T07:04:02.791654+0000","last_became_peered":"2024-04-22T07:04:02.791654+0000","last_unstale":"2024-04-22T07:04:30.034436+0000","last_undegraded":"2024-04-22T07:04:30.034436+0000","last_fullsized":"2024-04-22T07:04:30.034436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860230+0000","last_change":"2024-04-22T07:04:03.797391+0000","last_active":"2024-04-22T07:04:04.860230+0000","last_peered":"2024-04-22T07:04:04.860230+0000","last_clean":"2024-04-22T07:04:04.860230+0000","last_became_active":"2024-04-22T07:04:03.797123+0000","last_became_peered":"2024-04-22T07:04:03.797123+0000","last_unstale":"2024-04-22T07:04:04.860230+0000","last_undegraded":"2024-04-22T07:04:04.860230+0000","last_fullsized":"2024-04-22T07:04:04.860230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.833517+0000","last_change":"2024-04-22T07:04:02.793753+0000","last_active":"2024-04-22T07:04:13.833517+0000","last_peered":"2024-04-22T07:04:13.833517+0000","last_clean":"2024-04-22T07:04:13.833517+0000","last_became_active":"2024-04-22T07:04:02.793436+0000","last_became_peered":"2024-04-22T07:04:02.793436+0000","last_unstale":"2024-04-22T07:04:13.833517+0000","last_undegraded":"2024-04-22T07:04:13.833517+0000","last_fullsized":"2024-04-22T07:04:13.833517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790527+0000","last_change":"2024-04-22T07:04:03.800665+0000","last_active":"2024-04-22T07:04:04.790527+0000","last_peered":"2024-04-22T07:04:04.790527+0000","last_clean":"2024-04-22T07:04:04.790527+0000","last_became_active":"2024-04-22T07:04:03.800227+0000","last_became_peered":"2024-04-22T07:04:03.800227+0000","last_unstale":"2024-04-22T07:04:04.790527+0000","last_undegraded":"2024-04-22T07:04:04.790527+0000","last_fullsized":"2024-04-22T07:04:04.790527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'10","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.031633+0000","last_change":"2024-04-22T07:04:02.791309+0000","last_active":"2024-04-22T07:04:30.031633+0000","last_peered":"2024-04-22T07:04:30.031633+0000","last_clean":"2024-04-22T07:04:30.031633+0000","last_became_active":"2024-04-22T07:04:02.791000+0000","last_became_peered":"2024-04-22T07:04:02.791000+0000","last_unstale":"2024-04-22T07:04:30.031633+0000","last_undegraded":"2024-04-22T07:04:30.031633+0000","last_fullsized":"2024-04-22T07:04:30.031633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861543+0000","last_change":"2024-04-22T07:04:03.794316+0000","last_active":"2024-04-22T07:04:04.861543+0000","last_peered":"2024-04-22T07:04:04.861543+0000","last_clean":"2024-04-22T07:04:04.861543+0000","last_became_active":"2024-04-22T07:04:03.793646+0000","last_became_peered":"2024-04-22T07:04:03.793646+0000","last_unstale":"2024-04-22T07:04:04.861543+0000","last_undegraded":"2024-04-22T07:04:04.861543+0000","last_fullsized":"2024-04-22T07:04:04.861543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.033424+0000","last_change":"2024-04-22T07:04:02.791024+0000","last_active":"2024-04-22T07:04:30.033424+0000","last_peered":"2024-04-22T07:04:30.033424+0000","last_clean":"2024-04-22T07:04:30.033424+0000","last_became_active":"2024-04-22T07:04:02.789160+0000","last_became_peered":"2024-04-22T07:04:02.789160+0000","last_unstale":"2024-04-22T07:04:30.033424+0000","last_undegraded":"2024-04-22T07:04:30.033424+0000","last_fullsized":"2024-04-22T07:04:30.033424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859792+0000","last_change":"2024-04-22T07:04:03.798436+0000","last_active":"2024-04-22T07:04:04.859792+0000","last_peered":"2024-04-22T07:04:04.859792+0000","last_clean":"2024-04-22T07:04:04.859792+0000","last_became_active":"2024-04-22T07:04:03.797946+0000","last_became_peered":"2024-04-22T07:04:03.797946+0000","last_unstale":"2024-04-22T07:04:04.859792+0000","last_undegraded":"2024-04-22T07:04:04.859792+0000","last_fullsized":"2024-04-22T07:04:04.859792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.001390+0000","last_change":"2024-04-22T07:04:02.796551+0000","last_active":"2024-04-22T07:04:24.001390+0000","last_peered":"2024-04-22T07:04:24.001390+0000","last_clean":"2024-04-22T07:04:24.001390+0000","last_became_active":"2024-04-22T07:04:02.796332+0000","last_became_peered":"2024-04-22T07:04:02.796332+0000","last_unstale":"2024-04-22T07:04:24.001390+0000","last_undegraded":"2024-04-22T07:04:24.001390+0000","last_fullsized":"2024-04-22T07:04:24.001390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:15.854295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861758+0000","last_change":"2024-04-22T07:04:03.804622+0000","last_active":"2024-04-22T07:04:04.861758+0000","last_peered":"2024-04-22T07:04:04.861758+0000","last_clean":"2024-04-22T07:04:04.861758+0000","last_became_active":"2024-04-22T07:04:03.804324+0000","last_became_peered":"2024-04-22T07:04:03.804324+0000","last_unstale":"2024-04-22T07:04:04.861758+0000","last_undegraded":"2024-04-22T07:04:04.861758+0000","last_fullsized":"2024-04-22T07:04:04.861758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.002064+0000","last_change":"2024-04-22T07:04:02.787085+0000","last_active":"2024-04-22T07:04:24.002064+0000","last_peered":"2024-04-22T07:04:24.002064+0000","last_clean":"2024-04-22T07:04:24.002064+0000","last_became_active":"2024-04-22T07:04:02.786777+0000","last_became_peered":"2024-04-22T07:04:02.786777+0000","last_unstale":"2024-04-22T07:04:24.002064+0000","last_undegraded":"2024-04-22T07:04:24.002064+0000","last_fullsized":"2024-04-22T07:04:24.002064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860535+0000","last_change":"2024-04-22T07:04:03.796356+0000","last_active":"2024-04-22T07:04:04.860535+0000","last_peered":"2024-04-22T07:04:04.860535+0000","last_clean":"2024-04-22T07:04:04.860535+0000","last_became_active":"2024-04-22T07:04:03.796030+0000","last_became_peered":"2024-04-22T07:04:03.796030+0000","last_unstale":"2024-04-22T07:04:04.860535+0000","last_undegraded":"2024-04-22T07:04:04.860535+0000","last_fullsized":"2024-04-22T07:04:04.860535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.000472+0000","last_change":"2024-04-22T07:04:02.787425+0000","last_active":"2024-04-22T07:04:24.000472+0000","last_peered":"2024-04-22T07:04:24.000472+0000","last_clean":"2024-04-22T07:04:24.000472+0000","last_became_active":"2024-04-22T07:04:02.787146+0000","last_became_peered":"2024-04-22T07:04:02.787146+0000","last_unstale":"2024-04-22T07:04:24.000472+0000","last_undegraded":"2024-04-22T07:04:24.000472+0000","last_fullsized":"2024-04-22T07:04:24.000472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861631+0000","last_change":"2024-04-22T07:04:03.802924+0000","last_active":"2024-04-22T07:04:04.861631+0000","last_peered":"2024-04-22T07:04:04.861631+0000","last_clean":"2024-04-22T07:04:04.861631+0000","last_became_active":"2024-04-22T07:04:03.802644+0000","last_became_peered":"2024-04-22T07:04:03.802644+0000","last_unstale":"2024-04-22T07:04:04.861631+0000","last_undegraded":"2024-04-22T07:04:04.861631+0000","last_fullsized":"2024-04-22T07:04:04.861631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.783163+0000","last_change":"2024-04-22T07:04:02.794481+0000","last_active":"2024-04-22T07:04:08.783163+0000","last_peered":"2024-04-22T07:04:08.783163+0000","last_clean":"2024-04-22T07:04:08.783163+0000","last_became_active":"2024-04-22T07:04:02.793982+0000","last_became_peered":"2024-04-22T07:04:02.793982+0000","last_unstale":"2024-04-22T07:04:08.783163+0000","last_undegraded":"2024-04-22T07:04:08.783163+0000","last_fullsized":"2024-04-22T07:04:08.783163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:58.906416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862139+0000","last_change":"2024-04-22T07:04:03.802724+0000","last_active":"2024-04-22T07:04:04.862139+0000","last_peered":"2024-04-22T07:04:04.862139+0000","last_clean":"2024-04-22T07:04:04.862139+0000","last_became_active":"2024-04-22T07:04:03.802434+0000","last_became_peered":"2024-04-22T07:04:03.802434+0000","last_unstale":"2024-04-22T07:04:04.862139+0000","last_undegraded":"2024-04-22T07:04:04.862139+0000","last_fullsized":"2024-04-22T07:04:04.862139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.030841+0000","last_change":"2024-04-22T07:04:02.791375+0000","last_active":"2024-04-22T07:04:30.030841+0000","last_peered":"2024-04-22T07:04:30.030841+0000","last_clean":"2024-04-22T07:04:30.030841+0000","last_became_active":"2024-04-22T07:04:02.791095+0000","last_became_peered":"2024-04-22T07:04:02.791095+0000","last_unstale":"2024-04-22T07:04:30.030841+0000","last_undegraded":"2024-04-22T07:04:30.030841+0000","last_fullsized":"2024-04-22T07:04:30.030841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862265+0000","last_change":"2024-04-22T07:04:03.802028+0000","last_active":"2024-04-22T07:04:04.862265+0000","last_peered":"2024-04-22T07:04:04.862265+0000","last_clean":"2024-04-22T07:04:04.862265+0000","last_became_active":"2024-04-22T07:04:03.801715+0000","last_became_peered":"2024-04-22T07:04:03.801715+0000","last_unstale":"2024-04-22T07:04:04.862265+0000","last_undegraded":"2024-04-22T07:04:04.862265+0000","last_fullsized":"2024-04-22T07:04:04.862265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:29.815068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.002037+0000","last_change":"2024-04-22T07:04:02.793270+0000","last_active":"2024-04-22T07:04:24.002037+0000","last_peered":"2024-04-22T07:04:24.002037+0000","last_clean":"2024-04-22T07:04:24.002037+0000","last_became_active":"2024-04-22T07:04:02.793009+0000","last_became_peered":"2024-04-22T07:04:02.793009+0000","last_unstale":"2024-04-22T07:04:24.002037+0000","last_undegraded":"2024-04-22T07:04:24.002037+0000","last_fullsized":"2024-04-22T07:04:24.002037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861459+0000","last_change":"2024-04-22T07:04:03.800175+0000","last_active":"2024-04-22T07:04:04.861459+0000","last_peered":"2024-04-22T07:04:04.861459+0000","last_clean":"2024-04-22T07:04:04.861459+0000","last_became_active":"2024-04-22T07:04:03.799902+0000","last_became_peered":"2024-04-22T07:04:03.799902+0000","last_unstale":"2024-04-22T07:04:04.861459+0000","last_undegraded":"2024-04-22T07:04:04.861459+0000","last_fullsized":"2024-04-22T07:04:04.861459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.001001+0000","last_change":"2024-04-22T07:04:02.793643+0000","last_active":"2024-04-22T07:04:24.001001+0000","last_peered":"2024-04-22T07:04:24.001001+0000","last_clean":"2024-04-22T07:04:24.001001+0000","last_became_active":"2024-04-22T07:04:02.793333+0000","last_became_peered":"2024-04-22T07:04:02.793333+0000","last_unstale":"2024-04-22T07:04:24.001001+0000","last_undegraded":"2024-04-22T07:04:24.001001+0000","last_fullsized":"2024-04-22T07:04:24.001001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862019+0000","last_change":"2024-04-22T07:04:03.801821+0000","last_active":"2024-04-22T07:04:04.862019+0000","last_peered":"2024-04-22T07:04:04.862019+0000","last_clean":"2024-04-22T07:04:04.862019+0000","last_became_active":"2024-04-22T07:04:03.801510+0000","last_became_peered":"2024-04-22T07:04:03.801510+0000","last_unstale":"2024-04-22T07:04:04.862019+0000","last_undegraded":"2024-04-22T07:04:04.862019+0000","last_fullsized":"2024-04-22T07:04:04.862019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861241+0000","last_change":"2024-04-22T07:03:59.215285+0000","last_active":"2024-04-22T07:04:04.861241+0000","last_peered":"2024-04-22T07:04:04.861241+0000","last_clean":"2024-04-22T07:04:04.861241+0000","last_became_active":"2024-04-22T07:03:59.214275+0000","last_became_peered":"2024-04-22T07:03:59.214275+0000","last_unstale":"2024-04-22T07:04:04.861241+0000","last_undegraded":"2024-04-22T07:04:04.861241+0000","last_fullsized":"2024-04-22T07:04:04.861241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_clean_scrub_stamp":"2024-04-22T07:03:57.959761+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:55.105073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.000978+0000","last_change":"2024-04-22T07:04:02.793384+0000","last_active":"2024-04-22T07:04:24.000978+0000","last_peered":"2024-04-22T07:04:24.000978+0000","last_clean":"2024-04-22T07:04:24.000978+0000","last_became_active":"2024-04-22T07:04:02.793077+0000","last_became_peered":"2024-04-22T07:04:02.793077+0000","last_unstale":"2024-04-22T07:04:24.000978+0000","last_undegraded":"2024-04-22T07:04:24.000978+0000","last_fullsized":"2024-04-22T07:04:24.000978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860779+0000","last_change":"2024-04-22T07:04:03.798134+0000","last_active":"2024-04-22T07:04:04.860779+0000","last_peered":"2024-04-22T07:04:04.860779+0000","last_clean":"2024-04-22T07:04:04.860779+0000","last_became_active":"2024-04-22T07:04:03.797848+0000","last_became_peered":"2024-04-22T07:04:03.797848+0000","last_unstale":"2024-04-22T07:04:04.860779+0000","last_undegraded":"2024-04-22T07:04:04.860779+0000","last_fullsized":"2024-04-22T07:04:04.860779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.915772+0000","last_change":"2024-04-22T07:04:02.789591+0000","last_active":"2024-04-22T07:04:18.915772+0000","last_peered":"2024-04-22T07:04:18.915772+0000","last_clean":"2024-04-22T07:04:18.915772+0000","last_became_active":"2024-04-22T07:04:02.788785+0000","last_became_peered":"2024-04-22T07:04:02.788785+0000","last_unstale":"2024-04-22T07:04:18.915772+0000","last_undegraded":"2024-04-22T07:04:18.915772+0000","last_fullsized":"2024-04-22T07:04:18.915772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859611+0000","last_change":"2024-04-22T07:04:03.789757+0000","last_active":"2024-04-22T07:04:04.859611+0000","last_peered":"2024-04-22T07:04:04.859611+0000","last_clean":"2024-04-22T07:04:04.859611+0000","last_became_active":"2024-04-22T07:04:03.789463+0000","last_became_peered":"2024-04-22T07:04:03.789463+0000","last_unstale":"2024-04-22T07:04:04.859611+0000","last_undegraded":"2024-04-22T07:04:04.859611+0000","last_fullsized":"2024-04-22T07:04:04.859611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:16.859332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.915436+0000","last_change":"2024-04-22T07:04:02.790600+0000","last_active":"2024-04-22T07:04:18.915436+0000","last_peered":"2024-04-22T07:04:18.915436+0000","last_clean":"2024-04-22T07:04:18.915436+0000","last_became_active":"2024-04-22T07:04:02.790319+0000","last_became_peered":"2024-04-22T07:04:02.790319+0000","last_unstale":"2024-04-22T07:04:18.915436+0000","last_undegraded":"2024-04-22T07:04:18.915436+0000","last_fullsized":"2024-04-22T07:04:18.915436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787840+0000","last_change":"2024-04-22T07:04:03.788231+0000","last_active":"2024-04-22T07:04:04.787840+0000","last_peered":"2024-04-22T07:04:04.787840+0000","last_clean":"2024-04-22T07:04:04.787840+0000","last_became_active":"2024-04-22T07:04:03.787918+0000","last_became_peered":"2024-04-22T07:04:03.787918+0000","last_unstale":"2024-04-22T07:04:04.787840+0000","last_undegraded":"2024-04-22T07:04:04.787840+0000","last_fullsized":"2024-04-22T07:04:04.787840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.036874+0000","last_change":"2024-04-22T07:04:02.796245+0000","last_active":"2024-04-22T07:04:30.036874+0000","last_peered":"2024-04-22T07:04:30.036874+0000","last_clean":"2024-04-22T07:04:30.036874+0000","last_became_active":"2024-04-22T07:04:02.795947+0000","last_became_peered":"2024-04-22T07:04:02.795947+0000","last_unstale":"2024-04-22T07:04:30.036874+0000","last_undegraded":"2024-04-22T07:04:30.036874+0000","last_fullsized":"2024-04-22T07:04:30.036874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790274+0000","last_change":"2024-04-22T07:04:03.793330+0000","last_active":"2024-04-22T07:04:04.790274+0000","last_peered":"2024-04-22T07:04:04.790274+0000","last_clean":"2024-04-22T07:04:04.790274+0000","last_became_active":"2024-04-22T07:04:03.793037+0000","last_became_peered":"2024-04-22T07:04:03.793037+0000","last_unstale":"2024-04-22T07:04:04.790274+0000","last_undegraded":"2024-04-22T07:04:04.790274+0000","last_fullsized":"2024-04-22T07:04:04.790274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:13.368853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787998+0000","last_change":"2024-04-22T07:04:02.793498+0000","last_active":"2024-04-22T07:04:04.787998+0000","last_peered":"2024-04-22T07:04:04.787998+0000","last_clean":"2024-04-22T07:04:04.787998+0000","last_became_active":"2024-04-22T07:04:02.793215+0000","last_became_peered":"2024-04-22T07:04:02.793215+0000","last_unstale":"2024-04-22T07:04:04.787998+0000","last_undegraded":"2024-04-22T07:04:04.787998+0000","last_fullsized":"2024-04-22T07:04:04.787998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860866+0000","last_change":"2024-04-22T07:04:03.804152+0000","last_active":"2024-04-22T07:04:04.860866+0000","last_peered":"2024-04-22T07:04:04.860866+0000","last_clean":"2024-04-22T07:04:04.860866+0000","last_became_active":"2024-04-22T07:04:03.801961+0000","last_became_peered":"2024-04-22T07:04:03.801961+0000","last_unstale":"2024-04-22T07:04:04.860866+0000","last_undegraded":"2024-04-22T07:04:04.860866+0000","last_fullsized":"2024-04-22T07:04:04.860866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.004118+0000","last_change":"2024-04-22T07:04:02.793445+0000","last_active":"2024-04-22T07:04:24.004118+0000","last_peered":"2024-04-22T07:04:24.004118+0000","last_clean":"2024-04-22T07:04:24.004118+0000","last_became_active":"2024-04-22T07:04:02.793164+0000","last_became_peered":"2024-04-22T07:04:02.793164+0000","last_unstale":"2024-04-22T07:04:24.004118+0000","last_undegraded":"2024-04-22T07:04:24.004118+0000","last_fullsized":"2024-04-22T07:04:24.004118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788013+0000","last_change":"2024-04-22T07:04:03.789589+0000","last_active":"2024-04-22T07:04:04.788013+0000","last_peered":"2024-04-22T07:04:04.788013+0000","last_clean":"2024-04-22T07:04:04.788013+0000","last_became_active":"2024-04-22T07:04:03.789310+0000","last_became_peered":"2024-04-22T07:04:03.789310+0000","last_unstale":"2024-04-22T07:04:04.788013+0000","last_undegraded":"2024-04-22T07:04:04.788013+0000","last_fullsized":"2024-04-22T07:04:04.788013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.004469+0000","last_change":"2024-04-22T07:04:02.789243+0000","last_active":"2024-04-22T07:04:24.004469+0000","last_peered":"2024-04-22T07:04:24.004469+0000","last_clean":"2024-04-22T07:04:24.004469+0000","last_became_active":"2024-04-22T07:04:02.788897+0000","last_became_peered":"2024-04-22T07:04:02.788897+0000","last_unstale":"2024-04-22T07:04:24.004469+0000","last_undegraded":"2024-04-22T07:04:24.004469+0000","last_fullsized":"2024-04-22T07:04:24.004469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860272+0000","last_change":"2024-04-22T07:04:03.793656+0000","last_active":"2024-04-22T07:04:04.860272+0000","last_peered":"2024-04-22T07:04:04.860272+0000","last_clean":"2024-04-22T07:04:04.860272+0000","last_became_active":"2024-04-22T07:04:03.793311+0000","last_became_peered":"2024-04-22T07:04:03.793311+0000","last_unstale":"2024-04-22T07:04:04.860272+0000","last_undegraded":"2024-04-22T07:04:04.860272+0000","last_fullsized":"2024-04-22T07:04:04.860272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.032054+0000","last_change":"2024-04-22T07:04:02.786878+0000","last_active":"2024-04-22T07:04:30.032054+0000","last_peered":"2024-04-22T07:04:30.032054+0000","last_clean":"2024-04-22T07:04:30.032054+0000","last_became_active":"2024-04-22T07:04:02.786526+0000","last_became_peered":"2024-04-22T07:04:02.786526+0000","last_unstale":"2024-04-22T07:04:30.032054+0000","last_undegraded":"2024-04-22T07:04:30.032054+0000","last_fullsized":"2024-04-22T07:04:30.032054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860520+0000","last_change":"2024-04-22T07:04:03.791544+0000","last_active":"2024-04-22T07:04:04.860520+0000","last_peered":"2024-04-22T07:04:04.860520+0000","last_clean":"2024-04-22T07:04:04.860520+0000","last_became_active":"2024-04-22T07:04:03.791289+0000","last_became_peered":"2024-04-22T07:04:03.791289+0000","last_unstale":"2024-04-22T07:04:04.860520+0000","last_undegraded":"2024-04-22T07:04:04.860520+0000","last_fullsized":"2024-04-22T07:04:04.860520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.824240+0000","last_change":"2024-04-22T07:04:02.795178+0000","last_active":"2024-04-22T07:04:13.824240+0000","last_peered":"2024-04-22T07:04:13.824240+0000","last_clean":"2024-04-22T07:04:13.824240+0000","last_became_active":"2024-04-22T07:04:02.794795+0000","last_became_peered":"2024-04-22T07:04:02.794795+0000","last_unstale":"2024-04-22T07:04:13.824240+0000","last_undegraded":"2024-04-22T07:04:13.824240+0000","last_fullsized":"2024-04-22T07:04:13.824240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790163+0000","last_change":"2024-04-22T07:04:03.788560+0000","last_active":"2024-04-22T07:04:04.790163+0000","last_peered":"2024-04-22T07:04:04.790163+0000","last_clean":"2024-04-22T07:04:04.790163+0000","last_became_active":"2024-04-22T07:04:03.788278+0000","last_became_peered":"2024-04-22T07:04:03.788278+0000","last_unstale":"2024-04-22T07:04:04.790163+0000","last_undegraded":"2024-04-22T07:04:04.790163+0000","last_fullsized":"2024-04-22T07:04:04.790163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:37.492038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.034048+0000","last_change":"2024-04-22T07:04:02.788532+0000","last_active":"2024-04-22T07:04:30.034048+0000","last_peered":"2024-04-22T07:04:30.034048+0000","last_clean":"2024-04-22T07:04:30.034048+0000","last_became_active":"2024-04-22T07:04:02.788254+0000","last_became_peered":"2024-04-22T07:04:02.788254+0000","last_unstale":"2024-04-22T07:04:30.034048+0000","last_undegraded":"2024-04-22T07:04:30.034048+0000","last_fullsized":"2024-04-22T07:04:30.034048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859748+0000","last_change":"2024-04-22T07:04:03.790052+0000","last_active":"2024-04-22T07:04:04.859748+0000","last_peered":"2024-04-22T07:04:04.859748+0000","last_clean":"2024-04-22T07:04:04.859748+0000","last_became_active":"2024-04-22T07:04:03.789351+0000","last_became_peered":"2024-04-22T07:04:03.789351+0000","last_unstale":"2024-04-22T07:04:04.859748+0000","last_undegraded":"2024-04-22T07:04:04.859748+0000","last_fullsized":"2024-04-22T07:04:04.859748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.000434+0000","last_change":"2024-04-22T07:04:02.792774+0000","last_active":"2024-04-22T07:04:24.000434+0000","last_peered":"2024-04-22T07:04:24.000434+0000","last_clean":"2024-04-22T07:04:24.000434+0000","last_became_active":"2024-04-22T07:04:02.792282+0000","last_became_peered":"2024-04-22T07:04:02.792282+0000","last_unstale":"2024-04-22T07:04:24.000434+0000","last_undegraded":"2024-04-22T07:04:24.000434+0000","last_fullsized":"2024-04-22T07:04:24.000434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860691+0000","last_change":"2024-04-22T07:04:03.796133+0000","last_active":"2024-04-22T07:04:04.860691+0000","last_peered":"2024-04-22T07:04:04.860691+0000","last_clean":"2024-04-22T07:04:04.860691+0000","last_became_active":"2024-04-22T07:04:03.795854+0000","last_became_peered":"2024-04-22T07:04:03.795854+0000","last_unstale":"2024-04-22T07:04:04.860691+0000","last_undegraded":"2024-04-22T07:04:04.860691+0000","last_fullsized":"2024-04-22T07:04:04.860691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.822027+0000","last_change":"2024-04-22T07:04:02.789099+0000","last_active":"2024-04-22T07:04:13.822027+0000","last_peered":"2024-04-22T07:04:13.822027+0000","last_clean":"2024-04-22T07:04:13.822027+0000","last_became_active":"2024-04-22T07:04:02.788782+0000","last_became_peered":"2024-04-22T07:04:02.788782+0000","last_unstale":"2024-04-22T07:04:13.822027+0000","last_undegraded":"2024-04-22T07:04:13.822027+0000","last_fullsized":"2024-04-22T07:04:13.822027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860112+0000","last_change":"2024-04-22T07:04:03.799845+0000","last_active":"2024-04-22T07:04:04.860112+0000","last_peered":"2024-04-22T07:04:04.860112+0000","last_clean":"2024-04-22T07:04:04.860112+0000","last_became_active":"2024-04-22T07:04:03.799553+0000","last_became_peered":"2024-04-22T07:04:03.799553+0000","last_unstale":"2024-04-22T07:04:04.860112+0000","last_undegraded":"2024-04-22T07:04:04.860112+0000","last_fullsized":"2024-04-22T07:04:04.860112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860699+0000","last_change":"2024-04-22T07:04:03.799918+0000","last_active":"2024-04-22T07:04:04.860699+0000","last_peered":"2024-04-22T07:04:04.860699+0000","last_clean":"2024-04-22T07:04:04.860699+0000","last_became_active":"2024-04-22T07:04:03.797851+0000","last_became_peered":"2024-04-22T07:04:03.797851+0000","last_unstale":"2024-04-22T07:04:04.860699+0000","last_undegraded":"2024-04-22T07:04:04.860699+0000","last_fullsized":"2024-04-22T07:04:04.860699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.035109+0000","last_change":"2024-04-22T07:04:02.793627+0000","last_active":"2024-04-22T07:04:30.035109+0000","last_peered":"2024-04-22T07:04:30.035109+0000","last_clean":"2024-04-22T07:04:30.035109+0000","last_became_active":"2024-04-22T07:04:02.793341+0000","last_became_peered":"2024-04-22T07:04:02.793341+0000","last_unstale":"2024-04-22T07:04:30.035109+0000","last_undegraded":"2024-04-22T07:04:30.035109+0000","last_fullsized":"2024-04-22T07:04:30.035109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:21:07.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861294+0000","last_change":"2024-04-22T07:04:03.797768+0000","last_active":"2024-04-22T07:04:04.861294+0000","last_peered":"2024-04-22T07:04:04.861294+0000","last_clean":"2024-04-22T07:04:04.861294+0000","last_became_active":"2024-04-22T07:04:03.797437+0000","last_became_peered":"2024-04-22T07:04:03.797437+0000","last_unstale":"2024-04-22T07:04:04.861294+0000","last_undegraded":"2024-04-22T07:04:04.861294+0000","last_fullsized":"2024-04-22T07:04:04.861294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.033921+0000","last_change":"2024-04-22T07:04:02.789875+0000","last_active":"2024-04-22T07:04:30.033921+0000","last_peered":"2024-04-22T07:04:30.033921+0000","last_clean":"2024-04-22T07:04:30.033921+0000","last_became_active":"2024-04-22T07:04:02.788989+0000","last_became_peered":"2024-04-22T07:04:02.788989+0000","last_unstale":"2024-04-22T07:04:30.033921+0000","last_undegraded":"2024-04-22T07:04:30.033921+0000","last_fullsized":"2024-04-22T07:04:30.033921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860096+0000","last_change":"2024-04-22T07:04:03.797671+0000","last_active":"2024-04-22T07:04:04.860096+0000","last_peered":"2024-04-22T07:04:04.860096+0000","last_clean":"2024-04-22T07:04:04.860096+0000","last_became_active":"2024-04-22T07:04:03.797399+0000","last_became_peered":"2024-04-22T07:04:03.797399+0000","last_unstale":"2024-04-22T07:04:04.860096+0000","last_undegraded":"2024-04-22T07:04:04.860096+0000","last_fullsized":"2024-04-22T07:04:04.860096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.780760+0000","last_change":"2024-04-22T07:04:02.796141+0000","last_active":"2024-04-22T07:04:08.780760+0000","last_peered":"2024-04-22T07:04:08.780760+0000","last_clean":"2024-04-22T07:04:08.780760+0000","last_became_active":"2024-04-22T07:04:02.795821+0000","last_became_peered":"2024-04-22T07:04:02.795821+0000","last_unstale":"2024-04-22T07:04:08.780760+0000","last_undegraded":"2024-04-22T07:04:08.780760+0000","last_fullsized":"2024-04-22T07:04:08.780760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860371+0000","last_change":"2024-04-22T07:04:03.796729+0000","last_active":"2024-04-22T07:04:04.860371+0000","last_peered":"2024-04-22T07:04:04.860371+0000","last_clean":"2024-04-22T07:04:04.860371+0000","last_became_active":"2024-04-22T07:04:03.796486+0000","last_became_peered":"2024-04-22T07:04:03.796486+0000","last_unstale":"2024-04-22T07:04:04.860371+0000","last_undegraded":"2024-04-22T07:04:04.860371+0000","last_fullsized":"2024-04-22T07:04:04.860371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.033128+0000","last_change":"2024-04-22T07:04:02.793214+0000","last_active":"2024-04-22T07:04:30.033128+0000","last_peered":"2024-04-22T07:04:30.033128+0000","last_clean":"2024-04-22T07:04:30.033128+0000","last_became_active":"2024-04-22T07:04:02.792922+0000","last_became_peered":"2024-04-22T07:04:02.792922+0000","last_unstale":"2024-04-22T07:04:30.033128+0000","last_undegraded":"2024-04-22T07:04:30.033128+0000","last_fullsized":"2024-04-22T07:04:30.033128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":205,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":663402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":74866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":73370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":827392,"data_stored":662176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:41.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:41.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.757+0000 7fa153832640 1 Processor -- start 2024-04-22T07:04:41.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.757+0000 7fa153832640 1 -- start start 2024-04-22T07:04:41.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa153832640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa14c0c8210 0x7fa14c1594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa153832640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa14c0c4cc0 0x7fa14c0cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa153832640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14c0c6fb0 con 0x7fa14c14f450 2024-04-22T07:04:41.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa153832640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fa14c159020 con 0x7fa14c0c8210 2024-04-22T07:04:41.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa153832640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa14c0c5110 con 0x7fa14c0c4cc0 2024-04-22T07:04:41.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa150dcd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa14c0c8210 0x7fa14c1594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa150dcd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa14c0c8210 0x7fa14c1594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51506/0 (socket says 172.21.15.92:51506) 2024-04-22T07:04:41.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa150dcd640 1 -- 172.21.15.92:0/2386853599 learned_addr learned my addr 172.21.15.92:0/2386853599 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:41.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa151dcf640 1 --2- 172.21.15.92:0/2386853599 >> v2:172.21.15.145:3300/0 conn(0x7fa14c0c4cc0 0x7fa14c0cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa150dcd640 1 -- 172.21.15.92:0/2386853599 >> v2:172.21.15.145:3300/0 conn(0x7fa14c0c4cc0 msgr2=0x7fa14c0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:41.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa150dcd640 1 --2- 172.21.15.92:0/2386853599 >> v2:172.21.15.145:3300/0 conn(0x7fa14c0c4cc0 0x7fa14c0cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa150dcd640 1 -- 172.21.15.92:0/2386853599 >> v1:172.21.15.145:6789/0 conn(0x7fa14c14f450 legacy=0x7fa14c0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.758+0000 7fa150dcd640 1 -- 172.21.15.92:0/2386853599 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c0c7250 con 0x7fa14c0c8210 2024-04-22T07:04:41.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.759+0000 7fa150dcd640 1 --2- 172.21.15.92:0/2386853599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa14c0c8210 0x7fa14c1594b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa14c1591a0 tx=0x7fa13c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c7c083eb450481 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.759+0000 7fa143fff640 1 -- 172.21.15.92:0/2386853599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c034070 con 0x7fa14c0c8210 2024-04-22T07:04:41.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.759+0000 7fa143fff640 1 -- 172.21.15.92:0/2386853599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13c030860 con 0x7fa14c0c8210 2024-04-22T07:04:41.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.759+0000 7fa143fff640 1 -- 172.21.15.92:0/2386853599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c030c30 con 0x7fa14c0c8210 2024-04-22T07:04:41.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.760+0000 7fa153832640 1 -- 172.21.15.92:0/2386853599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa14c0c8210 msgr2=0x7fa14c1594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.760+0000 7fa153832640 1 --2- 172.21.15.92:0/2386853599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa14c0c8210 0x7fa14c1594b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa14c1591a0 tx=0x7fa13c02b9a0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.760+0000 7fa153832640 1 -- 172.21.15.92:0/2386853599 shutdown_connections 2024-04-22T07:04:41.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.760+0000 7fa153832640 1 --2- 172.21.15.92:0/2386853599 >> v2:172.21.15.145:3300/0 conn(0x7fa14c0c4cc0 0x7fa14c0cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.760+0000 7fa153832640 1 --2- 172.21.15.92:0/2386853599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa14c0c8210 0x7fa14c1594b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.760+0000 7fa153832640 1 -- 172.21.15.92:0/2386853599 >> 172.21.15.92:0/2386853599 conn(0x7fa14c06f8d0 msgr2=0x7fa14c06fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.760+0000 7fa153832640 1 -- 172.21.15.92:0/2386853599 shutdown_connections 2024-04-22T07:04:41.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.760+0000 7fa153832640 1 -- 172.21.15.92:0/2386853599 wait complete. 2024-04-22T07:04:41.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.761+0000 7fa153832640 1 Processor -- start 2024-04-22T07:04:41.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.761+0000 7fa153832640 1 -- start start 2024-04-22T07:04:41.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa153832640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa14c0c4cc0 0x7fa14c1e2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa153832640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa14c0c8210 0x7fa14c0c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa153832640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa14c14f450 0x7fa14c0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa153832640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c14f830 con 0x7fa14c0c4cc0 2024-04-22T07:04:41.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa153832640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c158cb0 con 0x7fa14c0c8210 2024-04-22T07:04:41.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa153832640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa14c158960 con 0x7fa14c14f450 2024-04-22T07:04:41.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa1515ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa14c0c4cc0 0x7fa14c1e2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa1515ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa14c0c4cc0 0x7fa14c1e2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51520/0 (socket says 172.21.15.92:51520) 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa1515ce640 1 -- 172.21.15.92:0/2563148424 learned_addr learned my addr 172.21.15.92:0/2563148424 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa151dcf640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa14c14f450 0x7fa14c0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa150dcd640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa14c0c8210 0x7fa14c0c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa1515ce640 1 -- 172.21.15.92:0/2563148424 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa14c14f450 msgr2=0x7fa14c0cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa1515ce640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa14c14f450 0x7fa14c0cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa1515ce640 1 -- 172.21.15.92:0/2563148424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa14c0c8210 msgr2=0x7fa14c0c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa1515ce640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa14c0c8210 0x7fa14c0c25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa1515ce640 1 -- 172.21.15.92:0/2563148424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c145250 con 0x7fa14c0c4cc0 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa150dcd640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa14c0c8210 0x7fa14c0c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa1515ce640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa14c0c4cc0 0x7fa14c1e2160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c760 tx=0x7fa14800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa141ffb640 1 -- 172.21.15.92:0/2563148424 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148016020 con 0x7fa14c0c4cc0 2024-04-22T07:04:41.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa141ffb640 1 -- 172.21.15.92:0/2563148424 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa148006d50 con 0x7fa14c0c4cc0 2024-04-22T07:04:41.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa153832640 1 -- 172.21.15.92:0/2563148424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14c073b90 con 0x7fa14c0c4cc0 2024-04-22T07:04:41.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.762+0000 7fa141ffb640 1 -- 172.21.15.92:0/2563148424 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148003bb0 con 0x7fa14c0c4cc0 2024-04-22T07:04:41.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.763+0000 7fa153832640 1 -- 172.21.15.92:0/2563148424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14c072de0 con 0x7fa14c0c4cc0 2024-04-22T07:04:41.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.764+0000 7fa153832640 1 -- 172.21.15.92:0/2563148424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa14c06eab0 con 0x7fa14c0c4cc0 2024-04-22T07:04:41.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.764+0000 7fa141ffb640 1 -- 172.21.15.92:0/2563148424 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa14800a850 con 0x7fa14c0c4cc0 2024-04-22T07:04:41.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.764+0000 7fa141ffb640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa13006c8f0 0x7fa13006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.764+0000 7fa141ffb640 1 -- 172.21.15.92:0/2563148424 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa14808b8a0 con 0x7fa14c0c4cc0 2024-04-22T07:04:41.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.764+0000 7fa150dcd640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa13006c8f0 0x7fa13006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.765+0000 7fa150dcd640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa13006c8f0 0x7fa13006edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa13c036000 tx=0x7fa13c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.769+0000 7fa141ffb640 1 -- 172.21.15.92:0/2563148424 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa14c06eab0 con 0x7fa14c0c4cc0 2024-04-22T07:04:41.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.944+0000 7fa153832640 1 -- 172.21.15.92:0/2563148424 --> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa14c1e2900 con 0x7fa13006c8f0 2024-04-22T07:04:41.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.950+0000 7fa141ffb640 1 -- 172.21.15.92:0/2563148424 <== mgr.4107 v2:172.21.15.92:6832/479317737 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251928 (secure 0 0 0) 0x7fa14c1e2900 con 0x7fa13006c8f0 2024-04-22T07:04:41.951 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:41.952 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-22T07:04:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.953+0000 7fa153832640 1 -- 172.21.15.92:0/2563148424 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa13006c8f0 msgr2=0x7fa13006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.953+0000 7fa153832640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa13006c8f0 0x7fa13006edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa13c036000 tx=0x7fa13c03b040 comp rx=0 tx=0).stop 2024-04-22T07:04:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.953+0000 7fa153832640 1 -- 172.21.15.92:0/2563148424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa14c0c4cc0 msgr2=0x7fa14c1e2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.953+0000 7fa153832640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa14c0c4cc0 0x7fa14c1e2160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c760 tx=0x7fa14800cc20 comp rx=0 tx=0).stop 2024-04-22T07:04:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.953+0000 7fa153832640 1 -- 172.21.15.92:0/2563148424 shutdown_connections 2024-04-22T07:04:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.953+0000 7fa153832640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa13006c8f0 0x7fa13006edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.953+0000 7fa153832640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa14c14f450 0x7fa14c0cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.953+0000 7fa153832640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa14c0c8210 0x7fa14c0c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.953+0000 7fa153832640 1 --2- 172.21.15.92:0/2563148424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa14c0c4cc0 0x7fa14c1e2160 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.953+0000 7fa153832640 1 -- 172.21.15.92:0/2563148424 >> 172.21.15.92:0/2563148424 conn(0x7fa14c06f8d0 msgr2=0x7fa14c14e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.953+0000 7fa153832640 1 -- 172.21.15.92:0/2563148424 shutdown_connections 2024-04-22T07:04:41.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:41.953+0000 7fa153832640 1 -- 172.21.15.92:0/2563148424 wait complete. 2024-04-22T07:04:41.962 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-22T07:04:41.940240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":364,"num_write_kb":2478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":3088384,"data_stored":1764276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002119"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862171+0000","last_change":"2024-04-22T07:04:03.796924+0000","last_active":"2024-04-22T07:04:04.862171+0000","last_peered":"2024-04-22T07:04:04.862171+0000","last_clean":"2024-04-22T07:04:04.862171+0000","last_became_active":"2024-04-22T07:04:03.796652+0000","last_became_peered":"2024-04-22T07:04:03.796652+0000","last_unstale":"2024-04-22T07:04:04.862171+0000","last_undegraded":"2024-04-22T07:04:04.862171+0000","last_fullsized":"2024-04-22T07:04:04.862171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:01.867906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788117+0000","last_change":"2024-04-22T07:04:03.793652+0000","last_active":"2024-04-22T07:04:04.788117+0000","last_peered":"2024-04-22T07:04:04.788117+0000","last_clean":"2024-04-22T07:04:04.788117+0000","last_became_active":"2024-04-22T07:04:03.793397+0000","last_became_peered":"2024-04-22T07:04:03.793397+0000","last_unstale":"2024-04-22T07:04:04.788117+0000","last_undegraded":"2024-04-22T07:04:04.788117+0000","last_fullsized":"2024-04-22T07:04:04.788117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859908+0000","last_change":"2024-04-22T07:04:03.788534+0000","last_active":"2024-04-22T07:04:04.859908+0000","last_peered":"2024-04-22T07:04:04.859908+0000","last_clean":"2024-04-22T07:04:04.859908+0000","last_became_active":"2024-04-22T07:04:03.787747+0000","last_became_peered":"2024-04-22T07:04:03.787747+0000","last_unstale":"2024-04-22T07:04:04.859908+0000","last_undegraded":"2024-04-22T07:04:04.859908+0000","last_fullsized":"2024-04-22T07:04:04.859908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861337+0000","last_change":"2024-04-22T07:04:03.800150+0000","last_active":"2024-04-22T07:04:04.861337+0000","last_peered":"2024-04-22T07:04:04.861337+0000","last_clean":"2024-04-22T07:04:04.861337+0000","last_became_active":"2024-04-22T07:04:03.799887+0000","last_became_peered":"2024-04-22T07:04:03.799887+0000","last_unstale":"2024-04-22T07:04:04.861337+0000","last_undegraded":"2024-04-22T07:04:04.861337+0000","last_fullsized":"2024-04-22T07:04:04.861337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861727+0000","last_change":"2024-04-22T07:04:03.798482+0000","last_active":"2024-04-22T07:04:04.861727+0000","last_peered":"2024-04-22T07:04:04.861727+0000","last_clean":"2024-04-22T07:04:04.861727+0000","last_became_active":"2024-04-22T07:04:03.798205+0000","last_became_peered":"2024-04-22T07:04:03.798205+0000","last_unstale":"2024-04-22T07:04:04.861727+0000","last_undegraded":"2024-04-22T07:04:04.861727+0000","last_fullsized":"2024-04-22T07:04:04.861727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861044+0000","last_change":"2024-04-22T07:04:03.797373+0000","last_active":"2024-04-22T07:04:04.861044+0000","last_peered":"2024-04-22T07:04:04.861044+0000","last_clean":"2024-04-22T07:04:04.861044+0000","last_became_active":"2024-04-22T07:04:03.797090+0000","last_became_peered":"2024-04-22T07:04:03.797090+0000","last_unstale":"2024-04-22T07:04:04.861044+0000","last_undegraded":"2024-04-22T07:04:04.861044+0000","last_fullsized":"2024-04-22T07:04:04.861044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861437+0000","last_change":"2024-04-22T07:04:03.793725+0000","last_active":"2024-04-22T07:04:04.861437+0000","last_peered":"2024-04-22T07:04:04.861437+0000","last_clean":"2024-04-22T07:04:04.861437+0000","last_became_active":"2024-04-22T07:04:03.793418+0000","last_became_peered":"2024-04-22T07:04:03.793418+0000","last_unstale":"2024-04-22T07:04:04.861437+0000","last_undegraded":"2024-04-22T07:04:04.861437+0000","last_fullsized":"2024-04-22T07:04:04.861437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859500+0000","last_change":"2024-04-22T07:04:03.795377+0000","last_active":"2024-04-22T07:04:04.859500+0000","last_peered":"2024-04-22T07:04:04.859500+0000","last_clean":"2024-04-22T07:04:04.859500+0000","last_became_active":"2024-04-22T07:04:03.795106+0000","last_became_peered":"2024-04-22T07:04:03.795106+0000","last_unstale":"2024-04-22T07:04:04.859500+0000","last_undegraded":"2024-04-22T07:04:04.859500+0000","last_fullsized":"2024-04-22T07:04:04.859500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862209+0000","last_change":"2024-04-22T07:04:03.797692+0000","last_active":"2024-04-22T07:04:04.862209+0000","last_peered":"2024-04-22T07:04:04.862209+0000","last_clean":"2024-04-22T07:04:04.862209+0000","last_became_active":"2024-04-22T07:04:03.797396+0000","last_became_peered":"2024-04-22T07:04:03.797396+0000","last_unstale":"2024-04-22T07:04:04.862209+0000","last_undegraded":"2024-04-22T07:04:04.862209+0000","last_fullsized":"2024-04-22T07:04:04.862209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:39.068857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860742+0000","last_change":"2024-04-22T07:04:03.797228+0000","last_active":"2024-04-22T07:04:04.860742+0000","last_peered":"2024-04-22T07:04:04.860742+0000","last_clean":"2024-04-22T07:04:04.860742+0000","last_became_active":"2024-04-22T07:04:03.796614+0000","last_became_peered":"2024-04-22T07:04:03.796614+0000","last_unstale":"2024-04-22T07:04:04.860742+0000","last_undegraded":"2024-04-22T07:04:04.860742+0000","last_fullsized":"2024-04-22T07:04:04.860742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860497+0000","last_change":"2024-04-22T07:04:03.799434+0000","last_active":"2024-04-22T07:04:04.860497+0000","last_peered":"2024-04-22T07:04:04.860497+0000","last_clean":"2024-04-22T07:04:04.860497+0000","last_became_active":"2024-04-22T07:04:03.799173+0000","last_became_peered":"2024-04-22T07:04:03.799173+0000","last_unstale":"2024-04-22T07:04:04.860497+0000","last_undegraded":"2024-04-22T07:04:04.860497+0000","last_fullsized":"2024-04-22T07:04:04.860497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790488+0000","last_change":"2024-04-22T07:04:03.800035+0000","last_active":"2024-04-22T07:04:04.790488+0000","last_peered":"2024-04-22T07:04:04.790488+0000","last_clean":"2024-04-22T07:04:04.790488+0000","last_became_active":"2024-04-22T07:04:03.799646+0000","last_became_peered":"2024-04-22T07:04:03.799646+0000","last_unstale":"2024-04-22T07:04:04.790488+0000","last_undegraded":"2024-04-22T07:04:04.790488+0000","last_fullsized":"2024-04-22T07:04:04.790488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:17.287068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860651+0000","last_change":"2024-04-22T07:04:03.797324+0000","last_active":"2024-04-22T07:04:04.860651+0000","last_peered":"2024-04-22T07:04:04.860651+0000","last_clean":"2024-04-22T07:04:04.860651+0000","last_became_active":"2024-04-22T07:04:03.797039+0000","last_became_peered":"2024-04-22T07:04:03.797039+0000","last_unstale":"2024-04-22T07:04:04.860651+0000","last_undegraded":"2024-04-22T07:04:04.860651+0000","last_fullsized":"2024-04-22T07:04:04.860651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859720+0000","last_change":"2024-04-22T07:04:03.790732+0000","last_active":"2024-04-22T07:04:04.859720+0000","last_peered":"2024-04-22T07:04:04.859720+0000","last_clean":"2024-04-22T07:04:04.859720+0000","last_became_active":"2024-04-22T07:04:03.788340+0000","last_became_peered":"2024-04-22T07:04:03.788340+0000","last_unstale":"2024-04-22T07:04:04.859720+0000","last_undegraded":"2024-04-22T07:04:04.859720+0000","last_fullsized":"2024-04-22T07:04:04.859720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787860+0000","last_change":"2024-04-22T07:04:03.799368+0000","last_active":"2024-04-22T07:04:04.787860+0000","last_peered":"2024-04-22T07:04:04.787860+0000","last_clean":"2024-04-22T07:04:04.787860+0000","last_became_active":"2024-04-22T07:04:03.799067+0000","last_became_peered":"2024-04-22T07:04:03.799067+0000","last_unstale":"2024-04-22T07:04:04.787860+0000","last_undegraded":"2024-04-22T07:04:04.787860+0000","last_fullsized":"2024-04-22T07:04:04.787860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860988+0000","last_change":"2024-04-22T07:04:03.802947+0000","last_active":"2024-04-22T07:04:04.860988+0000","last_peered":"2024-04-22T07:04:04.860988+0000","last_clean":"2024-04-22T07:04:04.860988+0000","last_became_active":"2024-04-22T07:04:03.802493+0000","last_became_peered":"2024-04-22T07:04:03.802493+0000","last_unstale":"2024-04-22T07:04:04.860988+0000","last_undegraded":"2024-04-22T07:04:04.860988+0000","last_fullsized":"2024-04-22T07:04:04.860988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860714+0000","last_change":"2024-04-22T07:04:03.795581+0000","last_active":"2024-04-22T07:04:04.860714+0000","last_peered":"2024-04-22T07:04:04.860714+0000","last_clean":"2024-04-22T07:04:04.860714+0000","last_became_active":"2024-04-22T07:04:03.795292+0000","last_became_peered":"2024-04-22T07:04:03.795292+0000","last_unstale":"2024-04-22T07:04:04.860714+0000","last_undegraded":"2024-04-22T07:04:04.860714+0000","last_fullsized":"2024-04-22T07:04:04.860714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859993+0000","last_change":"2024-04-22T07:04:03.796132+0000","last_active":"2024-04-22T07:04:04.859993+0000","last_peered":"2024-04-22T07:04:04.859993+0000","last_clean":"2024-04-22T07:04:04.859993+0000","last_became_active":"2024-04-22T07:04:03.795069+0000","last_became_peered":"2024-04-22T07:04:03.795069+0000","last_unstale":"2024-04-22T07:04:04.859993+0000","last_undegraded":"2024-04-22T07:04:04.859993+0000","last_fullsized":"2024-04-22T07:04:04.859993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:08.934970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862125+0000","last_change":"2024-04-22T07:04:03.794552+0000","last_active":"2024-04-22T07:04:04.862125+0000","last_peered":"2024-04-22T07:04:04.862125+0000","last_clean":"2024-04-22T07:04:04.862125+0000","last_became_active":"2024-04-22T07:04:03.794046+0000","last_became_peered":"2024-04-22T07:04:03.794046+0000","last_unstale":"2024-04-22T07:04:04.862125+0000","last_undegraded":"2024-04-22T07:04:04.862125+0000","last_fullsized":"2024-04-22T07:04:04.862125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860703+0000","last_change":"2024-04-22T07:04:03.798686+0000","last_active":"2024-04-22T07:04:04.860703+0000","last_peered":"2024-04-22T07:04:04.860703+0000","last_clean":"2024-04-22T07:04:04.860703+0000","last_became_active":"2024-04-22T07:04:03.798104+0000","last_became_peered":"2024-04-22T07:04:03.798104+0000","last_unstale":"2024-04-22T07:04:04.860703+0000","last_undegraded":"2024-04-22T07:04:04.860703+0000","last_fullsized":"2024-04-22T07:04:04.860703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860494+0000","last_change":"2024-04-22T07:04:03.794215+0000","last_active":"2024-04-22T07:04:04.860494+0000","last_peered":"2024-04-22T07:04:04.860494+0000","last_clean":"2024-04-22T07:04:04.860494+0000","last_became_active":"2024-04-22T07:04:03.793990+0000","last_became_peered":"2024-04-22T07:04:03.793990+0000","last_unstale":"2024-04-22T07:04:04.860494+0000","last_undegraded":"2024-04-22T07:04:04.860494+0000","last_fullsized":"2024-04-22T07:04:04.860494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861162+0000","last_change":"2024-04-22T07:04:03.798539+0000","last_active":"2024-04-22T07:04:04.861162+0000","last_peered":"2024-04-22T07:04:04.861162+0000","last_clean":"2024-04-22T07:04:04.861162+0000","last_became_active":"2024-04-22T07:04:03.797713+0000","last_became_peered":"2024-04-22T07:04:03.797713+0000","last_unstale":"2024-04-22T07:04:04.861162+0000","last_undegraded":"2024-04-22T07:04:04.861162+0000","last_fullsized":"2024-04-22T07:04:04.861162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859945+0000","last_change":"2024-04-22T07:04:03.798610+0000","last_active":"2024-04-22T07:04:04.859945+0000","last_peered":"2024-04-22T07:04:04.859945+0000","last_clean":"2024-04-22T07:04:04.859945+0000","last_became_active":"2024-04-22T07:04:03.797981+0000","last_became_peered":"2024-04-22T07:04:03.797981+0000","last_unstale":"2024-04-22T07:04:04.859945+0000","last_undegraded":"2024-04-22T07:04:04.859945+0000","last_fullsized":"2024-04-22T07:04:04.859945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861507+0000","last_change":"2024-04-22T07:04:03.802742+0000","last_active":"2024-04-22T07:04:04.861507+0000","last_peered":"2024-04-22T07:04:04.861507+0000","last_clean":"2024-04-22T07:04:04.861507+0000","last_became_active":"2024-04-22T07:04:03.802127+0000","last_became_peered":"2024-04-22T07:04:03.802127+0000","last_unstale":"2024-04-22T07:04:04.861507+0000","last_undegraded":"2024-04-22T07:04:04.861507+0000","last_fullsized":"2024-04-22T07:04:04.861507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860138+0000","last_change":"2024-04-22T07:04:03.798789+0000","last_active":"2024-04-22T07:04:04.860138+0000","last_peered":"2024-04-22T07:04:04.860138+0000","last_clean":"2024-04-22T07:04:04.860138+0000","last_became_active":"2024-04-22T07:04:03.798505+0000","last_became_peered":"2024-04-22T07:04:03.798505+0000","last_unstale":"2024-04-22T07:04:04.860138+0000","last_undegraded":"2024-04-22T07:04:04.860138+0000","last_fullsized":"2024-04-22T07:04:04.860138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860919+0000","last_change":"2024-04-22T07:04:03.794111+0000","last_active":"2024-04-22T07:04:04.860919+0000","last_peered":"2024-04-22T07:04:04.860919+0000","last_clean":"2024-04-22T07:04:04.860919+0000","last_became_active":"2024-04-22T07:04:03.793682+0000","last_became_peered":"2024-04-22T07:04:03.793682+0000","last_unstale":"2024-04-22T07:04:04.860919+0000","last_undegraded":"2024-04-22T07:04:04.860919+0000","last_fullsized":"2024-04-22T07:04:04.860919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788083+0000","last_change":"2024-04-22T07:04:03.790167+0000","last_active":"2024-04-22T07:04:04.788083+0000","last_peered":"2024-04-22T07:04:04.788083+0000","last_clean":"2024-04-22T07:04:04.788083+0000","last_became_active":"2024-04-22T07:04:03.789873+0000","last_became_peered":"2024-04-22T07:04:03.789873+0000","last_unstale":"2024-04-22T07:04:04.788083+0000","last_undegraded":"2024-04-22T07:04:04.788083+0000","last_fullsized":"2024-04-22T07:04:04.788083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859954+0000","last_change":"2024-04-22T07:04:03.798713+0000","last_active":"2024-04-22T07:04:04.859954+0000","last_peered":"2024-04-22T07:04:04.859954+0000","last_clean":"2024-04-22T07:04:04.859954+0000","last_became_active":"2024-04-22T07:04:03.798344+0000","last_became_peered":"2024-04-22T07:04:03.798344+0000","last_unstale":"2024-04-22T07:04:04.859954+0000","last_undegraded":"2024-04-22T07:04:04.859954+0000","last_fullsized":"2024-04-22T07:04:04.859954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860634+0000","last_change":"2024-04-22T07:04:03.795452+0000","last_active":"2024-04-22T07:04:04.860634+0000","last_peered":"2024-04-22T07:04:04.860634+0000","last_clean":"2024-04-22T07:04:04.860634+0000","last_became_active":"2024-04-22T07:04:03.794970+0000","last_became_peered":"2024-04-22T07:04:03.794970+0000","last_unstale":"2024-04-22T07:04:04.860634+0000","last_undegraded":"2024-04-22T07:04:04.860634+0000","last_fullsized":"2024-04-22T07:04:04.860634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861203+0000","last_change":"2024-04-22T07:04:03.799581+0000","last_active":"2024-04-22T07:04:04.861203+0000","last_peered":"2024-04-22T07:04:04.861203+0000","last_clean":"2024-04-22T07:04:04.861203+0000","last_became_active":"2024-04-22T07:04:03.799328+0000","last_became_peered":"2024-04-22T07:04:03.799328+0000","last_unstale":"2024-04-22T07:04:04.861203+0000","last_undegraded":"2024-04-22T07:04:04.861203+0000","last_fullsized":"2024-04-22T07:04:04.861203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859944+0000","last_change":"2024-04-22T07:04:03.796830+0000","last_active":"2024-04-22T07:04:04.859944+0000","last_peered":"2024-04-22T07:04:04.859944+0000","last_clean":"2024-04-22T07:04:04.859944+0000","last_became_active":"2024-04-22T07:04:03.796551+0000","last_became_peered":"2024-04-22T07:04:03.796551+0000","last_unstale":"2024-04-22T07:04:04.859944+0000","last_undegraded":"2024-04-22T07:04:04.859944+0000","last_fullsized":"2024-04-22T07:04:04.859944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860101+0000","last_change":"2024-04-22T07:04:03.795320+0000","last_active":"2024-04-22T07:04:04.860101+0000","last_peered":"2024-04-22T07:04:04.860101+0000","last_clean":"2024-04-22T07:04:04.860101+0000","last_became_active":"2024-04-22T07:04:03.795025+0000","last_became_peered":"2024-04-22T07:04:03.795025+0000","last_unstale":"2024-04-22T07:04:04.860101+0000","last_undegraded":"2024-04-22T07:04:04.860101+0000","last_fullsized":"2024-04-22T07:04:04.860101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790119+0000","last_change":"2024-04-22T07:04:03.788859+0000","last_active":"2024-04-22T07:04:04.790119+0000","last_peered":"2024-04-22T07:04:04.790119+0000","last_clean":"2024-04-22T07:04:04.790119+0000","last_became_active":"2024-04-22T07:04:03.788573+0000","last_became_peered":"2024-04-22T07:04:03.788573+0000","last_unstale":"2024-04-22T07:04:04.790119+0000","last_undegraded":"2024-04-22T07:04:04.790119+0000","last_fullsized":"2024-04-22T07:04:04.790119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:36.716689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.826359+0000","last_change":"2024-04-22T07:04:02.795724+0000","last_active":"2024-04-22T07:04:13.826359+0000","last_peered":"2024-04-22T07:04:13.826359+0000","last_clean":"2024-04-22T07:04:13.826359+0000","last_became_active":"2024-04-22T07:04:02.795371+0000","last_became_peered":"2024-04-22T07:04:02.795371+0000","last_unstale":"2024-04-22T07:04:13.826359+0000","last_undegraded":"2024-04-22T07:04:13.826359+0000","last_fullsized":"2024-04-22T07:04:13.826359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860689+0000","last_change":"2024-04-22T07:04:03.787030+0000","last_active":"2024-04-22T07:04:04.860689+0000","last_peered":"2024-04-22T07:04:04.860689+0000","last_clean":"2024-04-22T07:04:04.860689+0000","last_became_active":"2024-04-22T07:04:03.786357+0000","last_became_peered":"2024-04-22T07:04:03.786357+0000","last_unstale":"2024-04-22T07:04:04.860689+0000","last_undegraded":"2024-04-22T07:04:04.860689+0000","last_fullsized":"2024-04-22T07:04:04.860689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.004655+0000","last_change":"2024-04-22T07:04:02.791429+0000","last_active":"2024-04-22T07:04:24.004655+0000","last_peered":"2024-04-22T07:04:24.004655+0000","last_clean":"2024-04-22T07:04:24.004655+0000","last_became_active":"2024-04-22T07:04:02.791114+0000","last_became_peered":"2024-04-22T07:04:02.791114+0000","last_unstale":"2024-04-22T07:04:24.004655+0000","last_undegraded":"2024-04-22T07:04:24.004655+0000","last_fullsized":"2024-04-22T07:04:24.004655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.917309+0000","last_change":"2024-04-22T07:04:02.789531+0000","last_active":"2024-04-22T07:04:18.917309+0000","last_peered":"2024-04-22T07:04:18.917309+0000","last_clean":"2024-04-22T07:04:18.917309+0000","last_became_active":"2024-04-22T07:04:02.789248+0000","last_became_peered":"2024-04-22T07:04:02.789248+0000","last_unstale":"2024-04-22T07:04:18.917309+0000","last_undegraded":"2024-04-22T07:04:18.917309+0000","last_fullsized":"2024-04-22T07:04:18.917309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:06:43.031782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859624+0000","last_change":"2024-04-22T07:04:03.790142+0000","last_active":"2024-04-22T07:04:04.859624+0000","last_peered":"2024-04-22T07:04:04.859624+0000","last_clean":"2024-04-22T07:04:04.859624+0000","last_became_active":"2024-04-22T07:04:03.789869+0000","last_became_peered":"2024-04-22T07:04:03.789869+0000","last_unstale":"2024-04-22T07:04:04.859624+0000","last_undegraded":"2024-04-22T07:04:04.859624+0000","last_fullsized":"2024-04-22T07:04:04.859624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.002089+0000","last_change":"2024-04-22T07:04:02.790537+0000","last_active":"2024-04-22T07:04:24.002089+0000","last_peered":"2024-04-22T07:04:24.002089+0000","last_clean":"2024-04-22T07:04:24.002089+0000","last_became_active":"2024-04-22T07:04:02.789072+0000","last_became_peered":"2024-04-22T07:04:02.789072+0000","last_unstale":"2024-04-22T07:04:24.002089+0000","last_undegraded":"2024-04-22T07:04:24.002089+0000","last_fullsized":"2024-04-22T07:04:24.002089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787835+0000","last_change":"2024-04-22T07:04:03.791239+0000","last_active":"2024-04-22T07:04:04.787835+0000","last_peered":"2024-04-22T07:04:04.787835+0000","last_clean":"2024-04-22T07:04:04.787835+0000","last_became_active":"2024-04-22T07:04:03.790515+0000","last_became_peered":"2024-04-22T07:04:03.790515+0000","last_unstale":"2024-04-22T07:04:04.787835+0000","last_undegraded":"2024-04-22T07:04:04.787835+0000","last_fullsized":"2024-04-22T07:04:04.787835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.825000+0000","last_change":"2024-04-22T07:04:02.793582+0000","last_active":"2024-04-22T07:04:13.825000+0000","last_peered":"2024-04-22T07:04:13.825000+0000","last_clean":"2024-04-22T07:04:13.825000+0000","last_became_active":"2024-04-22T07:04:02.793281+0000","last_became_peered":"2024-04-22T07:04:02.793281+0000","last_unstale":"2024-04-22T07:04:13.825000+0000","last_undegraded":"2024-04-22T07:04:13.825000+0000","last_fullsized":"2024-04-22T07:04:13.825000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860510+0000","last_change":"2024-04-22T07:04:03.793994+0000","last_active":"2024-04-22T07:04:04.860510+0000","last_peered":"2024-04-22T07:04:04.860510+0000","last_clean":"2024-04-22T07:04:04.860510+0000","last_became_active":"2024-04-22T07:04:03.793734+0000","last_became_peered":"2024-04-22T07:04:03.793734+0000","last_unstale":"2024-04-22T07:04:04.860510+0000","last_undegraded":"2024-04-22T07:04:04.860510+0000","last_fullsized":"2024-04-22T07:04:04.860510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.034436+0000","last_change":"2024-04-22T07:04:02.791945+0000","last_active":"2024-04-22T07:04:30.034436+0000","last_peered":"2024-04-22T07:04:30.034436+0000","last_clean":"2024-04-22T07:04:30.034436+0000","last_became_active":"2024-04-22T07:04:02.791654+0000","last_became_peered":"2024-04-22T07:04:02.791654+0000","last_unstale":"2024-04-22T07:04:30.034436+0000","last_undegraded":"2024-04-22T07:04:30.034436+0000","last_fullsized":"2024-04-22T07:04:30.034436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860230+0000","last_change":"2024-04-22T07:04:03.797391+0000","last_active":"2024-04-22T07:04:04.860230+0000","last_peered":"2024-04-22T07:04:04.860230+0000","last_clean":"2024-04-22T07:04:04.860230+0000","last_became_active":"2024-04-22T07:04:03.797123+0000","last_became_peered":"2024-04-22T07:04:03.797123+0000","last_unstale":"2024-04-22T07:04:04.860230+0000","last_undegraded":"2024-04-22T07:04:04.860230+0000","last_fullsized":"2024-04-22T07:04:04.860230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.833517+0000","last_change":"2024-04-22T07:04:02.793753+0000","last_active":"2024-04-22T07:04:13.833517+0000","last_peered":"2024-04-22T07:04:13.833517+0000","last_clean":"2024-04-22T07:04:13.833517+0000","last_became_active":"2024-04-22T07:04:02.793436+0000","last_became_peered":"2024-04-22T07:04:02.793436+0000","last_unstale":"2024-04-22T07:04:13.833517+0000","last_undegraded":"2024-04-22T07:04:13.833517+0000","last_fullsized":"2024-04-22T07:04:13.833517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790527+0000","last_change":"2024-04-22T07:04:03.800665+0000","last_active":"2024-04-22T07:04:04.790527+0000","last_peered":"2024-04-22T07:04:04.790527+0000","last_clean":"2024-04-22T07:04:04.790527+0000","last_became_active":"2024-04-22T07:04:03.800227+0000","last_became_peered":"2024-04-22T07:04:03.800227+0000","last_unstale":"2024-04-22T07:04:04.790527+0000","last_undegraded":"2024-04-22T07:04:04.790527+0000","last_fullsized":"2024-04-22T07:04:04.790527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'10","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.031633+0000","last_change":"2024-04-22T07:04:02.791309+0000","last_active":"2024-04-22T07:04:30.031633+0000","last_peered":"2024-04-22T07:04:30.031633+0000","last_clean":"2024-04-22T07:04:30.031633+0000","last_became_active":"2024-04-22T07:04:02.791000+0000","last_became_peered":"2024-04-22T07:04:02.791000+0000","last_unstale":"2024-04-22T07:04:30.031633+0000","last_undegraded":"2024-04-22T07:04:30.031633+0000","last_fullsized":"2024-04-22T07:04:30.031633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861543+0000","last_change":"2024-04-22T07:04:03.794316+0000","last_active":"2024-04-22T07:04:04.861543+0000","last_peered":"2024-04-22T07:04:04.861543+0000","last_clean":"2024-04-22T07:04:04.861543+0000","last_became_active":"2024-04-22T07:04:03.793646+0000","last_became_peered":"2024-04-22T07:04:03.793646+0000","last_unstale":"2024-04-22T07:04:04.861543+0000","last_undegraded":"2024-04-22T07:04:04.861543+0000","last_fullsized":"2024-04-22T07:04:04.861543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.033424+0000","last_change":"2024-04-22T07:04:02.791024+0000","last_active":"2024-04-22T07:04:30.033424+0000","last_peered":"2024-04-22T07:04:30.033424+0000","last_clean":"2024-04-22T07:04:30.033424+0000","last_became_active":"2024-04-22T07:04:02.789160+0000","last_became_peered":"2024-04-22T07:04:02.789160+0000","last_unstale":"2024-04-22T07:04:30.033424+0000","last_undegraded":"2024-04-22T07:04:30.033424+0000","last_fullsized":"2024-04-22T07:04:30.033424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859792+0000","last_change":"2024-04-22T07:04:03.798436+0000","last_active":"2024-04-22T07:04:04.859792+0000","last_peered":"2024-04-22T07:04:04.859792+0000","last_clean":"2024-04-22T07:04:04.859792+0000","last_became_active":"2024-04-22T07:04:03.797946+0000","last_became_peered":"2024-04-22T07:04:03.797946+0000","last_unstale":"2024-04-22T07:04:04.859792+0000","last_undegraded":"2024-04-22T07:04:04.859792+0000","last_fullsized":"2024-04-22T07:04:04.859792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.001390+0000","last_change":"2024-04-22T07:04:02.796551+0000","last_active":"2024-04-22T07:04:24.001390+0000","last_peered":"2024-04-22T07:04:24.001390+0000","last_clean":"2024-04-22T07:04:24.001390+0000","last_became_active":"2024-04-22T07:04:02.796332+0000","last_became_peered":"2024-04-22T07:04:02.796332+0000","last_unstale":"2024-04-22T07:04:24.001390+0000","last_undegraded":"2024-04-22T07:04:24.001390+0000","last_fullsized":"2024-04-22T07:04:24.001390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:15.854295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861758+0000","last_change":"2024-04-22T07:04:03.804622+0000","last_active":"2024-04-22T07:04:04.861758+0000","last_peered":"2024-04-22T07:04:04.861758+0000","last_clean":"2024-04-22T07:04:04.861758+0000","last_became_active":"2024-04-22T07:04:03.804324+0000","last_became_peered":"2024-04-22T07:04:03.804324+0000","last_unstale":"2024-04-22T07:04:04.861758+0000","last_undegraded":"2024-04-22T07:04:04.861758+0000","last_fullsized":"2024-04-22T07:04:04.861758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.002064+0000","last_change":"2024-04-22T07:04:02.787085+0000","last_active":"2024-04-22T07:04:24.002064+0000","last_peered":"2024-04-22T07:04:24.002064+0000","last_clean":"2024-04-22T07:04:24.002064+0000","last_became_active":"2024-04-22T07:04:02.786777+0000","last_became_peered":"2024-04-22T07:04:02.786777+0000","last_unstale":"2024-04-22T07:04:24.002064+0000","last_undegraded":"2024-04-22T07:04:24.002064+0000","last_fullsized":"2024-04-22T07:04:24.002064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860535+0000","last_change":"2024-04-22T07:04:03.796356+0000","last_active":"2024-04-22T07:04:04.860535+0000","last_peered":"2024-04-22T07:04:04.860535+0000","last_clean":"2024-04-22T07:04:04.860535+0000","last_became_active":"2024-04-22T07:04:03.796030+0000","last_became_peered":"2024-04-22T07:04:03.796030+0000","last_unstale":"2024-04-22T07:04:04.860535+0000","last_undegraded":"2024-04-22T07:04:04.860535+0000","last_fullsized":"2024-04-22T07:04:04.860535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.000472+0000","last_change":"2024-04-22T07:04:02.787425+0000","last_active":"2024-04-22T07:04:24.000472+0000","last_peered":"2024-04-22T07:04:24.000472+0000","last_clean":"2024-04-22T07:04:24.000472+0000","last_became_active":"2024-04-22T07:04:02.787146+0000","last_became_peered":"2024-04-22T07:04:02.787146+0000","last_unstale":"2024-04-22T07:04:24.000472+0000","last_undegraded":"2024-04-22T07:04:24.000472+0000","last_fullsized":"2024-04-22T07:04:24.000472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861631+0000","last_change":"2024-04-22T07:04:03.802924+0000","last_active":"2024-04-22T07:04:04.861631+0000","last_peered":"2024-04-22T07:04:04.861631+0000","last_clean":"2024-04-22T07:04:04.861631+0000","last_became_active":"2024-04-22T07:04:03.802644+0000","last_became_peered":"2024-04-22T07:04:03.802644+0000","last_unstale":"2024-04-22T07:04:04.861631+0000","last_undegraded":"2024-04-22T07:04:04.861631+0000","last_fullsized":"2024-04-22T07:04:04.861631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.783163+0000","last_change":"2024-04-22T07:04:02.794481+0000","last_active":"2024-04-22T07:04:08.783163+0000","last_peered":"2024-04-22T07:04:08.783163+0000","last_clean":"2024-04-22T07:04:08.783163+0000","last_became_active":"2024-04-22T07:04:02.793982+0000","last_became_peered":"2024-04-22T07:04:02.793982+0000","last_unstale":"2024-04-22T07:04:08.783163+0000","last_undegraded":"2024-04-22T07:04:08.783163+0000","last_fullsized":"2024-04-22T07:04:08.783163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:58.906416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862139+0000","last_change":"2024-04-22T07:04:03.802724+0000","last_active":"2024-04-22T07:04:04.862139+0000","last_peered":"2024-04-22T07:04:04.862139+0000","last_clean":"2024-04-22T07:04:04.862139+0000","last_became_active":"2024-04-22T07:04:03.802434+0000","last_became_peered":"2024-04-22T07:04:03.802434+0000","last_unstale":"2024-04-22T07:04:04.862139+0000","last_undegraded":"2024-04-22T07:04:04.862139+0000","last_fullsized":"2024-04-22T07:04:04.862139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.030841+0000","last_change":"2024-04-22T07:04:02.791375+0000","last_active":"2024-04-22T07:04:30.030841+0000","last_peered":"2024-04-22T07:04:30.030841+0000","last_clean":"2024-04-22T07:04:30.030841+0000","last_became_active":"2024-04-22T07:04:02.791095+0000","last_became_peered":"2024-04-22T07:04:02.791095+0000","last_unstale":"2024-04-22T07:04:30.030841+0000","last_undegraded":"2024-04-22T07:04:30.030841+0000","last_fullsized":"2024-04-22T07:04:30.030841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862265+0000","last_change":"2024-04-22T07:04:03.802028+0000","last_active":"2024-04-22T07:04:04.862265+0000","last_peered":"2024-04-22T07:04:04.862265+0000","last_clean":"2024-04-22T07:04:04.862265+0000","last_became_active":"2024-04-22T07:04:03.801715+0000","last_became_peered":"2024-04-22T07:04:03.801715+0000","last_unstale":"2024-04-22T07:04:04.862265+0000","last_undegraded":"2024-04-22T07:04:04.862265+0000","last_fullsized":"2024-04-22T07:04:04.862265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:29.815068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.002037+0000","last_change":"2024-04-22T07:04:02.793270+0000","last_active":"2024-04-22T07:04:24.002037+0000","last_peered":"2024-04-22T07:04:24.002037+0000","last_clean":"2024-04-22T07:04:24.002037+0000","last_became_active":"2024-04-22T07:04:02.793009+0000","last_became_peered":"2024-04-22T07:04:02.793009+0000","last_unstale":"2024-04-22T07:04:24.002037+0000","last_undegraded":"2024-04-22T07:04:24.002037+0000","last_fullsized":"2024-04-22T07:04:24.002037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861459+0000","last_change":"2024-04-22T07:04:03.800175+0000","last_active":"2024-04-22T07:04:04.861459+0000","last_peered":"2024-04-22T07:04:04.861459+0000","last_clean":"2024-04-22T07:04:04.861459+0000","last_became_active":"2024-04-22T07:04:03.799902+0000","last_became_peered":"2024-04-22T07:04:03.799902+0000","last_unstale":"2024-04-22T07:04:04.861459+0000","last_undegraded":"2024-04-22T07:04:04.861459+0000","last_fullsized":"2024-04-22T07:04:04.861459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.001001+0000","last_change":"2024-04-22T07:04:02.793643+0000","last_active":"2024-04-22T07:04:24.001001+0000","last_peered":"2024-04-22T07:04:24.001001+0000","last_clean":"2024-04-22T07:04:24.001001+0000","last_became_active":"2024-04-22T07:04:02.793333+0000","last_became_peered":"2024-04-22T07:04:02.793333+0000","last_unstale":"2024-04-22T07:04:24.001001+0000","last_undegraded":"2024-04-22T07:04:24.001001+0000","last_fullsized":"2024-04-22T07:04:24.001001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862019+0000","last_change":"2024-04-22T07:04:03.801821+0000","last_active":"2024-04-22T07:04:04.862019+0000","last_peered":"2024-04-22T07:04:04.862019+0000","last_clean":"2024-04-22T07:04:04.862019+0000","last_became_active":"2024-04-22T07:04:03.801510+0000","last_became_peered":"2024-04-22T07:04:03.801510+0000","last_unstale":"2024-04-22T07:04:04.862019+0000","last_undegraded":"2024-04-22T07:04:04.862019+0000","last_fullsized":"2024-04-22T07:04:04.862019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861241+0000","last_change":"2024-04-22T07:03:59.215285+0000","last_active":"2024-04-22T07:04:04.861241+0000","last_peered":"2024-04-22T07:04:04.861241+0000","last_clean":"2024-04-22T07:04:04.861241+0000","last_became_active":"2024-04-22T07:03:59.214275+0000","last_became_peered":"2024-04-22T07:03:59.214275+0000","last_unstale":"2024-04-22T07:04:04.861241+0000","last_undegraded":"2024-04-22T07:04:04.861241+0000","last_fullsized":"2024-04-22T07:04:04.861241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_clean_scrub_stamp":"2024-04-22T07:03:57.959761+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:55.105073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.000978+0000","last_change":"2024-04-22T07:04:02.793384+0000","last_active":"2024-04-22T07:04:24.000978+0000","last_peered":"2024-04-22T07:04:24.000978+0000","last_clean":"2024-04-22T07:04:24.000978+0000","last_became_active":"2024-04-22T07:04:02.793077+0000","last_became_peered":"2024-04-22T07:04:02.793077+0000","last_unstale":"2024-04-22T07:04:24.000978+0000","last_undegraded":"2024-04-22T07:04:24.000978+0000","last_fullsized":"2024-04-22T07:04:24.000978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860779+0000","last_change":"2024-04-22T07:04:03.798134+0000","last_active":"2024-04-22T07:04:04.860779+0000","last_peered":"2024-04-22T07:04:04.860779+0000","last_clean":"2024-04-22T07:04:04.860779+0000","last_became_active":"2024-04-22T07:04:03.797848+0000","last_became_peered":"2024-04-22T07:04:03.797848+0000","last_unstale":"2024-04-22T07:04:04.860779+0000","last_undegraded":"2024-04-22T07:04:04.860779+0000","last_fullsized":"2024-04-22T07:04:04.860779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.915772+0000","last_change":"2024-04-22T07:04:02.789591+0000","last_active":"2024-04-22T07:04:18.915772+0000","last_peered":"2024-04-22T07:04:18.915772+0000","last_clean":"2024-04-22T07:04:18.915772+0000","last_became_active":"2024-04-22T07:04:02.788785+0000","last_became_peered":"2024-04-22T07:04:02.788785+0000","last_unstale":"2024-04-22T07:04:18.915772+0000","last_undegraded":"2024-04-22T07:04:18.915772+0000","last_fullsized":"2024-04-22T07:04:18.915772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859611+0000","last_change":"2024-04-22T07:04:03.789757+0000","last_active":"2024-04-22T07:04:04.859611+0000","last_peered":"2024-04-22T07:04:04.859611+0000","last_clean":"2024-04-22T07:04:04.859611+0000","last_became_active":"2024-04-22T07:04:03.789463+0000","last_became_peered":"2024-04-22T07:04:03.789463+0000","last_unstale":"2024-04-22T07:04:04.859611+0000","last_undegraded":"2024-04-22T07:04:04.859611+0000","last_fullsized":"2024-04-22T07:04:04.859611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:16.859332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.915436+0000","last_change":"2024-04-22T07:04:02.790600+0000","last_active":"2024-04-22T07:04:18.915436+0000","last_peered":"2024-04-22T07:04:18.915436+0000","last_clean":"2024-04-22T07:04:18.915436+0000","last_became_active":"2024-04-22T07:04:02.790319+0000","last_became_peered":"2024-04-22T07:04:02.790319+0000","last_unstale":"2024-04-22T07:04:18.915436+0000","last_undegraded":"2024-04-22T07:04:18.915436+0000","last_fullsized":"2024-04-22T07:04:18.915436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787840+0000","last_change":"2024-04-22T07:04:03.788231+0000","last_active":"2024-04-22T07:04:04.787840+0000","last_peered":"2024-04-22T07:04:04.787840+0000","last_clean":"2024-04-22T07:04:04.787840+0000","last_became_active":"2024-04-22T07:04:03.787918+0000","last_became_peered":"2024-04-22T07:04:03.787918+0000","last_unstale":"2024-04-22T07:04:04.787840+0000","last_undegraded":"2024-04-22T07:04:04.787840+0000","last_fullsized":"2024-04-22T07:04:04.787840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.036874+0000","last_change":"2024-04-22T07:04:02.796245+0000","last_active":"2024-04-22T07:04:30.036874+0000","last_peered":"2024-04-22T07:04:30.036874+0000","last_clean":"2024-04-22T07:04:30.036874+0000","last_became_active":"2024-04-22T07:04:02.795947+0000","last_became_peered":"2024-04-22T07:04:02.795947+0000","last_unstale":"2024-04-22T07:04:30.036874+0000","last_undegraded":"2024-04-22T07:04:30.036874+0000","last_fullsized":"2024-04-22T07:04:30.036874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790274+0000","last_change":"2024-04-22T07:04:03.793330+0000","last_active":"2024-04-22T07:04:04.790274+0000","last_peered":"2024-04-22T07:04:04.790274+0000","last_clean":"2024-04-22T07:04:04.790274+0000","last_became_active":"2024-04-22T07:04:03.793037+0000","last_became_peered":"2024-04-22T07:04:03.793037+0000","last_unstale":"2024-04-22T07:04:04.790274+0000","last_undegraded":"2024-04-22T07:04:04.790274+0000","last_fullsized":"2024-04-22T07:04:04.790274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:13.368853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787998+0000","last_change":"2024-04-22T07:04:02.793498+0000","last_active":"2024-04-22T07:04:04.787998+0000","last_peered":"2024-04-22T07:04:04.787998+0000","last_clean":"2024-04-22T07:04:04.787998+0000","last_became_active":"2024-04-22T07:04:02.793215+0000","last_became_peered":"2024-04-22T07:04:02.793215+0000","last_unstale":"2024-04-22T07:04:04.787998+0000","last_undegraded":"2024-04-22T07:04:04.787998+0000","last_fullsized":"2024-04-22T07:04:04.787998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860866+0000","last_change":"2024-04-22T07:04:03.804152+0000","last_active":"2024-04-22T07:04:04.860866+0000","last_peered":"2024-04-22T07:04:04.860866+0000","last_clean":"2024-04-22T07:04:04.860866+0000","last_became_active":"2024-04-22T07:04:03.801961+0000","last_became_peered":"2024-04-22T07:04:03.801961+0000","last_unstale":"2024-04-22T07:04:04.860866+0000","last_undegraded":"2024-04-22T07:04:04.860866+0000","last_fullsized":"2024-04-22T07:04:04.860866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.004118+0000","last_change":"2024-04-22T07:04:02.793445+0000","last_active":"2024-04-22T07:04:24.004118+0000","last_peered":"2024-04-22T07:04:24.004118+0000","last_clean":"2024-04-22T07:04:24.004118+0000","last_became_active":"2024-04-22T07:04:02.793164+0000","last_became_peered":"2024-04-22T07:04:02.793164+0000","last_unstale":"2024-04-22T07:04:24.004118+0000","last_undegraded":"2024-04-22T07:04:24.004118+0000","last_fullsized":"2024-04-22T07:04:24.004118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788013+0000","last_change":"2024-04-22T07:04:03.789589+0000","last_active":"2024-04-22T07:04:04.788013+0000","last_peered":"2024-04-22T07:04:04.788013+0000","last_clean":"2024-04-22T07:04:04.788013+0000","last_became_active":"2024-04-22T07:04:03.789310+0000","last_became_peered":"2024-04-22T07:04:03.789310+0000","last_unstale":"2024-04-22T07:04:04.788013+0000","last_undegraded":"2024-04-22T07:04:04.788013+0000","last_fullsized":"2024-04-22T07:04:04.788013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.004469+0000","last_change":"2024-04-22T07:04:02.789243+0000","last_active":"2024-04-22T07:04:24.004469+0000","last_peered":"2024-04-22T07:04:24.004469+0000","last_clean":"2024-04-22T07:04:24.004469+0000","last_became_active":"2024-04-22T07:04:02.788897+0000","last_became_peered":"2024-04-22T07:04:02.788897+0000","last_unstale":"2024-04-22T07:04:24.004469+0000","last_undegraded":"2024-04-22T07:04:24.004469+0000","last_fullsized":"2024-04-22T07:04:24.004469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860272+0000","last_change":"2024-04-22T07:04:03.793656+0000","last_active":"2024-04-22T07:04:04.860272+0000","last_peered":"2024-04-22T07:04:04.860272+0000","last_clean":"2024-04-22T07:04:04.860272+0000","last_became_active":"2024-04-22T07:04:03.793311+0000","last_became_peered":"2024-04-22T07:04:03.793311+0000","last_unstale":"2024-04-22T07:04:04.860272+0000","last_undegraded":"2024-04-22T07:04:04.860272+0000","last_fullsized":"2024-04-22T07:04:04.860272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.032054+0000","last_change":"2024-04-22T07:04:02.786878+0000","last_active":"2024-04-22T07:04:30.032054+0000","last_peered":"2024-04-22T07:04:30.032054+0000","last_clean":"2024-04-22T07:04:30.032054+0000","last_became_active":"2024-04-22T07:04:02.786526+0000","last_became_peered":"2024-04-22T07:04:02.786526+0000","last_unstale":"2024-04-22T07:04:30.032054+0000","last_undegraded":"2024-04-22T07:04:30.032054+0000","last_fullsized":"2024-04-22T07:04:30.032054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860520+0000","last_change":"2024-04-22T07:04:03.791544+0000","last_active":"2024-04-22T07:04:04.860520+0000","last_peered":"2024-04-22T07:04:04.860520+0000","last_clean":"2024-04-22T07:04:04.860520+0000","last_became_active":"2024-04-22T07:04:03.791289+0000","last_became_peered":"2024-04-22T07:04:03.791289+0000","last_unstale":"2024-04-22T07:04:04.860520+0000","last_undegraded":"2024-04-22T07:04:04.860520+0000","last_fullsized":"2024-04-22T07:04:04.860520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.824240+0000","last_change":"2024-04-22T07:04:02.795178+0000","last_active":"2024-04-22T07:04:13.824240+0000","last_peered":"2024-04-22T07:04:13.824240+0000","last_clean":"2024-04-22T07:04:13.824240+0000","last_became_active":"2024-04-22T07:04:02.794795+0000","last_became_peered":"2024-04-22T07:04:02.794795+0000","last_unstale":"2024-04-22T07:04:13.824240+0000","last_undegraded":"2024-04-22T07:04:13.824240+0000","last_fullsized":"2024-04-22T07:04:13.824240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790163+0000","last_change":"2024-04-22T07:04:03.788560+0000","last_active":"2024-04-22T07:04:04.790163+0000","last_peered":"2024-04-22T07:04:04.790163+0000","last_clean":"2024-04-22T07:04:04.790163+0000","last_became_active":"2024-04-22T07:04:03.788278+0000","last_became_peered":"2024-04-22T07:04:03.788278+0000","last_unstale":"2024-04-22T07:04:04.790163+0000","last_undegraded":"2024-04-22T07:04:04.790163+0000","last_fullsized":"2024-04-22T07:04:04.790163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:37.492038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.034048+0000","last_change":"2024-04-22T07:04:02.788532+0000","last_active":"2024-04-22T07:04:30.034048+0000","last_peered":"2024-04-22T07:04:30.034048+0000","last_clean":"2024-04-22T07:04:30.034048+0000","last_became_active":"2024-04-22T07:04:02.788254+0000","last_became_peered":"2024-04-22T07:04:02.788254+0000","last_unstale":"2024-04-22T07:04:30.034048+0000","last_undegraded":"2024-04-22T07:04:30.034048+0000","last_fullsized":"2024-04-22T07:04:30.034048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859748+0000","last_change":"2024-04-22T07:04:03.790052+0000","last_active":"2024-04-22T07:04:04.859748+0000","last_peered":"2024-04-22T07:04:04.859748+0000","last_clean":"2024-04-22T07:04:04.859748+0000","last_became_active":"2024-04-22T07:04:03.789351+0000","last_became_peered":"2024-04-22T07:04:03.789351+0000","last_unstale":"2024-04-22T07:04:04.859748+0000","last_undegraded":"2024-04-22T07:04:04.859748+0000","last_fullsized":"2024-04-22T07:04:04.859748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.000434+0000","last_change":"2024-04-22T07:04:02.792774+0000","last_active":"2024-04-22T07:04:24.000434+0000","last_peered":"2024-04-22T07:04:24.000434+0000","last_clean":"2024-04-22T07:04:24.000434+0000","last_became_active":"2024-04-22T07:04:02.792282+0000","last_became_peered":"2024-04-22T07:04:02.792282+0000","last_unstale":"2024-04-22T07:04:24.000434+0000","last_undegraded":"2024-04-22T07:04:24.000434+0000","last_fullsized":"2024-04-22T07:04:24.000434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860691+0000","last_change":"2024-04-22T07:04:03.796133+0000","last_active":"2024-04-22T07:04:04.860691+0000","last_peered":"2024-04-22T07:04:04.860691+0000","last_clean":"2024-04-22T07:04:04.860691+0000","last_became_active":"2024-04-22T07:04:03.795854+0000","last_became_peered":"2024-04-22T07:04:03.795854+0000","last_unstale":"2024-04-22T07:04:04.860691+0000","last_undegraded":"2024-04-22T07:04:04.860691+0000","last_fullsized":"2024-04-22T07:04:04.860691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.822027+0000","last_change":"2024-04-22T07:04:02.789099+0000","last_active":"2024-04-22T07:04:13.822027+0000","last_peered":"2024-04-22T07:04:13.822027+0000","last_clean":"2024-04-22T07:04:13.822027+0000","last_became_active":"2024-04-22T07:04:02.788782+0000","last_became_peered":"2024-04-22T07:04:02.788782+0000","last_unstale":"2024-04-22T07:04:13.822027+0000","last_undegraded":"2024-04-22T07:04:13.822027+0000","last_fullsized":"2024-04-22T07:04:13.822027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860112+0000","last_change":"2024-04-22T07:04:03.799845+0000","last_active":"2024-04-22T07:04:04.860112+0000","last_peered":"2024-04-22T07:04:04.860112+0000","last_clean":"2024-04-22T07:04:04.860112+0000","last_became_active":"2024-04-22T07:04:03.799553+0000","last_became_peered":"2024-04-22T07:04:03.799553+0000","last_unstale":"2024-04-22T07:04:04.860112+0000","last_undegraded":"2024-04-22T07:04:04.860112+0000","last_fullsized":"2024-04-22T07:04:04.860112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860699+0000","last_change":"2024-04-22T07:04:03.799918+0000","last_active":"2024-04-22T07:04:04.860699+0000","last_peered":"2024-04-22T07:04:04.860699+0000","last_clean":"2024-04-22T07:04:04.860699+0000","last_became_active":"2024-04-22T07:04:03.797851+0000","last_became_peered":"2024-04-22T07:04:03.797851+0000","last_unstale":"2024-04-22T07:04:04.860699+0000","last_undegraded":"2024-04-22T07:04:04.860699+0000","last_fullsized":"2024-04-22T07:04:04.860699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.035109+0000","last_change":"2024-04-22T07:04:02.793627+0000","last_active":"2024-04-22T07:04:30.035109+0000","last_peered":"2024-04-22T07:04:30.035109+0000","last_clean":"2024-04-22T07:04:30.035109+0000","last_became_active":"2024-04-22T07:04:02.793341+0000","last_became_peered":"2024-04-22T07:04:02.793341+0000","last_unstale":"2024-04-22T07:04:30.035109+0000","last_undegraded":"2024-04-22T07:04:30.035109+0000","last_fullsized":"2024-04-22T07:04:30.035109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:21:07.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861294+0000","last_change":"2024-04-22T07:04:03.797768+0000","last_active":"2024-04-22T07:04:04.861294+0000","last_peered":"2024-04-22T07:04:04.861294+0000","last_clean":"2024-04-22T07:04:04.861294+0000","last_became_active":"2024-04-22T07:04:03.797437+0000","last_became_peered":"2024-04-22T07:04:03.797437+0000","last_unstale":"2024-04-22T07:04:04.861294+0000","last_undegraded":"2024-04-22T07:04:04.861294+0000","last_fullsized":"2024-04-22T07:04:04.861294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.033921+0000","last_change":"2024-04-22T07:04:02.789875+0000","last_active":"2024-04-22T07:04:30.033921+0000","last_peered":"2024-04-22T07:04:30.033921+0000","last_clean":"2024-04-22T07:04:30.033921+0000","last_became_active":"2024-04-22T07:04:02.788989+0000","last_became_peered":"2024-04-22T07:04:02.788989+0000","last_unstale":"2024-04-22T07:04:30.033921+0000","last_undegraded":"2024-04-22T07:04:30.033921+0000","last_fullsized":"2024-04-22T07:04:30.033921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860096+0000","last_change":"2024-04-22T07:04:03.797671+0000","last_active":"2024-04-22T07:04:04.860096+0000","last_peered":"2024-04-22T07:04:04.860096+0000","last_clean":"2024-04-22T07:04:04.860096+0000","last_became_active":"2024-04-22T07:04:03.797399+0000","last_became_peered":"2024-04-22T07:04:03.797399+0000","last_unstale":"2024-04-22T07:04:04.860096+0000","last_undegraded":"2024-04-22T07:04:04.860096+0000","last_fullsized":"2024-04-22T07:04:04.860096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.780760+0000","last_change":"2024-04-22T07:04:02.796141+0000","last_active":"2024-04-22T07:04:08.780760+0000","last_peered":"2024-04-22T07:04:08.780760+0000","last_clean":"2024-04-22T07:04:08.780760+0000","last_became_active":"2024-04-22T07:04:02.795821+0000","last_became_peered":"2024-04-22T07:04:02.795821+0000","last_unstale":"2024-04-22T07:04:08.780760+0000","last_undegraded":"2024-04-22T07:04:08.780760+0000","last_fullsized":"2024-04-22T07:04:08.780760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860371+0000","last_change":"2024-04-22T07:04:03.796729+0000","last_active":"2024-04-22T07:04:04.860371+0000","last_peered":"2024-04-22T07:04:04.860371+0000","last_clean":"2024-04-22T07:04:04.860371+0000","last_became_active":"2024-04-22T07:04:03.796486+0000","last_became_peered":"2024-04-22T07:04:03.796486+0000","last_unstale":"2024-04-22T07:04:04.860371+0000","last_undegraded":"2024-04-22T07:04:04.860371+0000","last_fullsized":"2024-04-22T07:04:04.860371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.033128+0000","last_change":"2024-04-22T07:04:02.793214+0000","last_active":"2024-04-22T07:04:30.033128+0000","last_peered":"2024-04-22T07:04:30.033128+0000","last_clean":"2024-04-22T07:04:30.033128+0000","last_became_active":"2024-04-22T07:04:02.792922+0000","last_became_peered":"2024-04-22T07:04:02.792922+0000","last_unstale":"2024-04-22T07:04:30.033128+0000","last_undegraded":"2024-04-22T07:04:30.033128+0000","last_fullsized":"2024-04-22T07:04:30.033128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":205,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":663402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":74866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":73370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":827392,"data_stored":662176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:41.966 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-22T07:04:41.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:42.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.102+0000 7f8177766640 1 Processor -- start 2024-04-22T07:04:42.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.103+0000 7f8177766640 1 -- start start 2024-04-22T07:04:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.103+0000 7f8177766640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f817006c7d0 0x7f8170154a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.103+0000 7f8177766640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f817006b970 0x7f8170154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.103+0000 7f8177766640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f817015eeb0 con 0x7f817006c3f0 2024-04-22T07:04:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.103+0000 7f8177766640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f817006fad0 con 0x7f817006c7d0 2024-04-22T07:04:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.103+0000 7f8177766640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f81700c9f60 con 0x7f817006b970 2024-04-22T07:04:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.103+0000 7f8174d01640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f817006c7d0 0x7f8170154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.103+0000 7f8174d01640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f817006c7d0 0x7f8170154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51522/0 (socket says 172.21.15.92:51522) 2024-04-22T07:04:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.104+0000 7f8174d01640 1 -- 172.21.15.92:0/307905656 learned_addr learned my addr 172.21.15.92:0/307905656 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.104+0000 7f8175d03640 1 --2- 172.21.15.92:0/307905656 >> v2:172.21.15.145:3300/0 conn(0x7f817006b970 0x7f8170154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.104+0000 7f8174d01640 1 -- 172.21.15.92:0/307905656 >> v2:172.21.15.145:3300/0 conn(0x7f817006b970 msgr2=0x7f8170154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.104+0000 7f8174d01640 1 --2- 172.21.15.92:0/307905656 >> v2:172.21.15.145:3300/0 conn(0x7f817006b970 0x7f8170154fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.104+0000 7f8174d01640 1 -- 172.21.15.92:0/307905656 >> v1:172.21.15.145:6789/0 conn(0x7f817006c3f0 legacy=0x7f817006ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:42.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.104+0000 7f8174d01640 1 -- 172.21.15.92:0/307905656 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81700ca240 con 0x7f817006c7d0 2024-04-22T07:04:42.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.104+0000 7f8174d01640 1 --2- 172.21.15.92:0/307905656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817006c7d0 0x7f8170154a70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f81700c3880 tx=0x7f816c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6981d49e6f5645c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.105+0000 7f8163fff640 1 -- 172.21.15.92:0/307905656 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c035020 con 0x7f817006c7d0 2024-04-22T07:04:42.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.105+0000 7f8163fff640 1 -- 172.21.15.92:0/307905656 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f816c031870 con 0x7f817006c7d0 2024-04-22T07:04:42.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.105+0000 7f8163fff640 1 -- 172.21.15.92:0/307905656 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c031c40 con 0x7f817006c7d0 2024-04-22T07:04:42.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.105+0000 7f8177766640 1 -- 172.21.15.92:0/307905656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817006c7d0 msgr2=0x7f8170154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.105+0000 7f8177766640 1 --2- 172.21.15.92:0/307905656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817006c7d0 0x7f8170154a70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f81700c3880 tx=0x7f816c02b910 comp rx=0 tx=0).stop 2024-04-22T07:04:42.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.105+0000 7f8177766640 1 -- 172.21.15.92:0/307905656 shutdown_connections 2024-04-22T07:04:42.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.105+0000 7f8177766640 1 --2- 172.21.15.92:0/307905656 >> v2:172.21.15.145:3300/0 conn(0x7f817006b970 0x7f8170154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.105+0000 7f8177766640 1 --2- 172.21.15.92:0/307905656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817006c7d0 0x7f8170154a70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.105+0000 7f8177766640 1 -- 172.21.15.92:0/307905656 >> 172.21.15.92:0/307905656 conn(0x7f817006e260 msgr2=0x7f81700715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.105+0000 7f8177766640 1 -- 172.21.15.92:0/307905656 shutdown_connections 2024-04-22T07:04:42.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.105+0000 7f8177766640 1 -- 172.21.15.92:0/307905656 wait complete. 2024-04-22T07:04:42.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.106+0000 7f8177766640 1 Processor -- start 2024-04-22T07:04:42.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.106+0000 7f8177766640 1 -- start start 2024-04-22T07:04:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8177766640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f817006b970 0x7f81700b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8177766640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f817006c3f0 0x7f81700cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8177766640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817006c7d0 0x7f81701f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8177766640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81700c8f70 con 0x7f817006c7d0 2024-04-22T07:04:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8177766640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81700c8df0 con 0x7f817006b970 2024-04-22T07:04:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8177766640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8170070670 con 0x7f817006c3f0 2024-04-22T07:04:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8175d03640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817006c7d0 0x7f81701f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8175d03640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817006c7d0 0x7f81701f0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51532/0 (socket says 172.21.15.92:51532) 2024-04-22T07:04:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8175d03640 1 -- 172.21.15.92:0/865498907 learned_addr learned my addr 172.21.15.92:0/865498907 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8174d01640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f817006c3f0 0x7f81700cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8175502640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f817006b970 0x7f81700b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8175d03640 1 -- 172.21.15.92:0/865498907 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f817006c3f0 msgr2=0x7f81700cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8175d03640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f817006c3f0 0x7f81700cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8175d03640 1 -- 172.21.15.92:0/865498907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f817006b970 msgr2=0x7f81700b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8175d03640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f817006b970 0x7f81700b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.107+0000 7f8175d03640 1 -- 172.21.15.92:0/865498907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81701598b0 con 0x7f817006c7d0 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.108+0000 7f8175d03640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817006c7d0 0x7f81701f0e70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8168007e70 tx=0x7f816800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.108+0000 7f8161ffb640 1 -- 172.21.15.92:0/865498907 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168014620 con 0x7f817006c7d0 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.108+0000 7f8161ffb640 1 -- 172.21.15.92:0/865498907 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8168019070 con 0x7f817006c7d0 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.108+0000 7f8177766640 1 -- 172.21.15.92:0/865498907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81701f06d0 con 0x7f817006c7d0 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.108+0000 7f8161ffb640 1 -- 172.21.15.92:0/865498907 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168014a20 con 0x7f817006c7d0 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.108+0000 7f8177766640 1 -- 172.21.15.92:0/865498907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81701f4a90 con 0x7f817006c7d0 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.109+0000 7f8177766640 1 -- 172.21.15.92:0/865498907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8170068b00 con 0x7f817006c7d0 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.109+0000 7f8161ffb640 1 -- 172.21.15.92:0/865498907 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8168014bc0 con 0x7f817006c7d0 2024-04-22T07:04:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.110+0000 7f8161ffb640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f815406c8f0 0x7f815406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.110+0000 7f8175502640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f815406c8f0 0x7f815406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.110+0000 7f8161ffb640 1 -- 172.21.15.92:0/865498907 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f816808c3d0 con 0x7f817006c7d0 2024-04-22T07:04:42.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.110+0000 7f8175502640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f815406c8f0 0x7f815406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8164004f60 tx=0x7f8164007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.114+0000 7f8161ffb640 1 -- 172.21.15.92:0/865498907 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81701f06d0 con 0x7f817006c7d0 2024-04-22T07:04:42.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.285+0000 7f8177766640 1 -- 172.21.15.92:0/865498907 --> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f817006cff0 con 0x7f815406c8f0 2024-04-22T07:04:42.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.291+0000 7f8161ffb640 1 -- 172.21.15.92:0/865498907 <== mgr.4107 v2:172.21.15.92:6832/479317737 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251928 (secure 0 0 0) 0x7f817006cff0 con 0x7f815406c8f0 2024-04-22T07:04:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:42.292 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-04-22T07:04:42.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.294+0000 7f8177766640 1 -- 172.21.15.92:0/865498907 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f815406c8f0 msgr2=0x7f815406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.294+0000 7f8177766640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f815406c8f0 0x7f815406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8164004f60 tx=0x7f8164007e30 comp rx=0 tx=0).stop 2024-04-22T07:04:42.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.294+0000 7f8177766640 1 -- 172.21.15.92:0/865498907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817006c7d0 msgr2=0x7f81701f0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.294+0000 7f8177766640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817006c7d0 0x7f81701f0e70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8168007e70 tx=0x7f816800e4b0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.294+0000 7f8177766640 1 -- 172.21.15.92:0/865498907 shutdown_connections 2024-04-22T07:04:42.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.294+0000 7f8177766640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f815406c8f0 0x7f815406edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.294+0000 7f8177766640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817006c7d0 0x7f81701f0e70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.294+0000 7f8177766640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f817006c3f0 0x7f81700cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.294+0000 7f8177766640 1 --2- 172.21.15.92:0/865498907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f817006b970 0x7f81700b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.294+0000 7f8177766640 1 -- 172.21.15.92:0/865498907 >> 172.21.15.92:0/865498907 conn(0x7f817006e260 msgr2=0x7f81700cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.294+0000 7f8177766640 1 -- 172.21.15.92:0/865498907 shutdown_connections 2024-04-22T07:04:42.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.294+0000 7f8177766640 1 -- 172.21.15.92:0/865498907 wait complete. 2024-04-22T07:04:42.302 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-22T07:04:41.940240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":364,"num_write_kb":2478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":3088384,"data_stored":1764276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002119"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862171+0000","last_change":"2024-04-22T07:04:03.796924+0000","last_active":"2024-04-22T07:04:04.862171+0000","last_peered":"2024-04-22T07:04:04.862171+0000","last_clean":"2024-04-22T07:04:04.862171+0000","last_became_active":"2024-04-22T07:04:03.796652+0000","last_became_peered":"2024-04-22T07:04:03.796652+0000","last_unstale":"2024-04-22T07:04:04.862171+0000","last_undegraded":"2024-04-22T07:04:04.862171+0000","last_fullsized":"2024-04-22T07:04:04.862171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:01.867906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788117+0000","last_change":"2024-04-22T07:04:03.793652+0000","last_active":"2024-04-22T07:04:04.788117+0000","last_peered":"2024-04-22T07:04:04.788117+0000","last_clean":"2024-04-22T07:04:04.788117+0000","last_became_active":"2024-04-22T07:04:03.793397+0000","last_became_peered":"2024-04-22T07:04:03.793397+0000","last_unstale":"2024-04-22T07:04:04.788117+0000","last_undegraded":"2024-04-22T07:04:04.788117+0000","last_fullsized":"2024-04-22T07:04:04.788117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859908+0000","last_change":"2024-04-22T07:04:03.788534+0000","last_active":"2024-04-22T07:04:04.859908+0000","last_peered":"2024-04-22T07:04:04.859908+0000","last_clean":"2024-04-22T07:04:04.859908+0000","last_became_active":"2024-04-22T07:04:03.787747+0000","last_became_peered":"2024-04-22T07:04:03.787747+0000","last_unstale":"2024-04-22T07:04:04.859908+0000","last_undegraded":"2024-04-22T07:04:04.859908+0000","last_fullsized":"2024-04-22T07:04:04.859908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861337+0000","last_change":"2024-04-22T07:04:03.800150+0000","last_active":"2024-04-22T07:04:04.861337+0000","last_peered":"2024-04-22T07:04:04.861337+0000","last_clean":"2024-04-22T07:04:04.861337+0000","last_became_active":"2024-04-22T07:04:03.799887+0000","last_became_peered":"2024-04-22T07:04:03.799887+0000","last_unstale":"2024-04-22T07:04:04.861337+0000","last_undegraded":"2024-04-22T07:04:04.861337+0000","last_fullsized":"2024-04-22T07:04:04.861337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861727+0000","last_change":"2024-04-22T07:04:03.798482+0000","last_active":"2024-04-22T07:04:04.861727+0000","last_peered":"2024-04-22T07:04:04.861727+0000","last_clean":"2024-04-22T07:04:04.861727+0000","last_became_active":"2024-04-22T07:04:03.798205+0000","last_became_peered":"2024-04-22T07:04:03.798205+0000","last_unstale":"2024-04-22T07:04:04.861727+0000","last_undegraded":"2024-04-22T07:04:04.861727+0000","last_fullsized":"2024-04-22T07:04:04.861727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861044+0000","last_change":"2024-04-22T07:04:03.797373+0000","last_active":"2024-04-22T07:04:04.861044+0000","last_peered":"2024-04-22T07:04:04.861044+0000","last_clean":"2024-04-22T07:04:04.861044+0000","last_became_active":"2024-04-22T07:04:03.797090+0000","last_became_peered":"2024-04-22T07:04:03.797090+0000","last_unstale":"2024-04-22T07:04:04.861044+0000","last_undegraded":"2024-04-22T07:04:04.861044+0000","last_fullsized":"2024-04-22T07:04:04.861044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861437+0000","last_change":"2024-04-22T07:04:03.793725+0000","last_active":"2024-04-22T07:04:04.861437+0000","last_peered":"2024-04-22T07:04:04.861437+0000","last_clean":"2024-04-22T07:04:04.861437+0000","last_became_active":"2024-04-22T07:04:03.793418+0000","last_became_peered":"2024-04-22T07:04:03.793418+0000","last_unstale":"2024-04-22T07:04:04.861437+0000","last_undegraded":"2024-04-22T07:04:04.861437+0000","last_fullsized":"2024-04-22T07:04:04.861437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859500+0000","last_change":"2024-04-22T07:04:03.795377+0000","last_active":"2024-04-22T07:04:04.859500+0000","last_peered":"2024-04-22T07:04:04.859500+0000","last_clean":"2024-04-22T07:04:04.859500+0000","last_became_active":"2024-04-22T07:04:03.795106+0000","last_became_peered":"2024-04-22T07:04:03.795106+0000","last_unstale":"2024-04-22T07:04:04.859500+0000","last_undegraded":"2024-04-22T07:04:04.859500+0000","last_fullsized":"2024-04-22T07:04:04.859500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862209+0000","last_change":"2024-04-22T07:04:03.797692+0000","last_active":"2024-04-22T07:04:04.862209+0000","last_peered":"2024-04-22T07:04:04.862209+0000","last_clean":"2024-04-22T07:04:04.862209+0000","last_became_active":"2024-04-22T07:04:03.797396+0000","last_became_peered":"2024-04-22T07:04:03.797396+0000","last_unstale":"2024-04-22T07:04:04.862209+0000","last_undegraded":"2024-04-22T07:04:04.862209+0000","last_fullsized":"2024-04-22T07:04:04.862209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:39.068857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860742+0000","last_change":"2024-04-22T07:04:03.797228+0000","last_active":"2024-04-22T07:04:04.860742+0000","last_peered":"2024-04-22T07:04:04.860742+0000","last_clean":"2024-04-22T07:04:04.860742+0000","last_became_active":"2024-04-22T07:04:03.796614+0000","last_became_peered":"2024-04-22T07:04:03.796614+0000","last_unstale":"2024-04-22T07:04:04.860742+0000","last_undegraded":"2024-04-22T07:04:04.860742+0000","last_fullsized":"2024-04-22T07:04:04.860742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860497+0000","last_change":"2024-04-22T07:04:03.799434+0000","last_active":"2024-04-22T07:04:04.860497+0000","last_peered":"2024-04-22T07:04:04.860497+0000","last_clean":"2024-04-22T07:04:04.860497+0000","last_became_active":"2024-04-22T07:04:03.799173+0000","last_became_peered":"2024-04-22T07:04:03.799173+0000","last_unstale":"2024-04-22T07:04:04.860497+0000","last_undegraded":"2024-04-22T07:04:04.860497+0000","last_fullsized":"2024-04-22T07:04:04.860497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790488+0000","last_change":"2024-04-22T07:04:03.800035+0000","last_active":"2024-04-22T07:04:04.790488+0000","last_peered":"2024-04-22T07:04:04.790488+0000","last_clean":"2024-04-22T07:04:04.790488+0000","last_became_active":"2024-04-22T07:04:03.799646+0000","last_became_peered":"2024-04-22T07:04:03.799646+0000","last_unstale":"2024-04-22T07:04:04.790488+0000","last_undegraded":"2024-04-22T07:04:04.790488+0000","last_fullsized":"2024-04-22T07:04:04.790488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:17.287068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860651+0000","last_change":"2024-04-22T07:04:03.797324+0000","last_active":"2024-04-22T07:04:04.860651+0000","last_peered":"2024-04-22T07:04:04.860651+0000","last_clean":"2024-04-22T07:04:04.860651+0000","last_became_active":"2024-04-22T07:04:03.797039+0000","last_became_peered":"2024-04-22T07:04:03.797039+0000","last_unstale":"2024-04-22T07:04:04.860651+0000","last_undegraded":"2024-04-22T07:04:04.860651+0000","last_fullsized":"2024-04-22T07:04:04.860651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859720+0000","last_change":"2024-04-22T07:04:03.790732+0000","last_active":"2024-04-22T07:04:04.859720+0000","last_peered":"2024-04-22T07:04:04.859720+0000","last_clean":"2024-04-22T07:04:04.859720+0000","last_became_active":"2024-04-22T07:04:03.788340+0000","last_became_peered":"2024-04-22T07:04:03.788340+0000","last_unstale":"2024-04-22T07:04:04.859720+0000","last_undegraded":"2024-04-22T07:04:04.859720+0000","last_fullsized":"2024-04-22T07:04:04.859720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787860+0000","last_change":"2024-04-22T07:04:03.799368+0000","last_active":"2024-04-22T07:04:04.787860+0000","last_peered":"2024-04-22T07:04:04.787860+0000","last_clean":"2024-04-22T07:04:04.787860+0000","last_became_active":"2024-04-22T07:04:03.799067+0000","last_became_peered":"2024-04-22T07:04:03.799067+0000","last_unstale":"2024-04-22T07:04:04.787860+0000","last_undegraded":"2024-04-22T07:04:04.787860+0000","last_fullsized":"2024-04-22T07:04:04.787860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860988+0000","last_change":"2024-04-22T07:04:03.802947+0000","last_active":"2024-04-22T07:04:04.860988+0000","last_peered":"2024-04-22T07:04:04.860988+0000","last_clean":"2024-04-22T07:04:04.860988+0000","last_became_active":"2024-04-22T07:04:03.802493+0000","last_became_peered":"2024-04-22T07:04:03.802493+0000","last_unstale":"2024-04-22T07:04:04.860988+0000","last_undegraded":"2024-04-22T07:04:04.860988+0000","last_fullsized":"2024-04-22T07:04:04.860988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860714+0000","last_change":"2024-04-22T07:04:03.795581+0000","last_active":"2024-04-22T07:04:04.860714+0000","last_peered":"2024-04-22T07:04:04.860714+0000","last_clean":"2024-04-22T07:04:04.860714+0000","last_became_active":"2024-04-22T07:04:03.795292+0000","last_became_peered":"2024-04-22T07:04:03.795292+0000","last_unstale":"2024-04-22T07:04:04.860714+0000","last_undegraded":"2024-04-22T07:04:04.860714+0000","last_fullsized":"2024-04-22T07:04:04.860714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:52:42.220506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859993+0000","last_change":"2024-04-22T07:04:03.796132+0000","last_active":"2024-04-22T07:04:04.859993+0000","last_peered":"2024-04-22T07:04:04.859993+0000","last_clean":"2024-04-22T07:04:04.859993+0000","last_became_active":"2024-04-22T07:04:03.795069+0000","last_became_peered":"2024-04-22T07:04:03.795069+0000","last_unstale":"2024-04-22T07:04:04.859993+0000","last_undegraded":"2024-04-22T07:04:04.859993+0000","last_fullsized":"2024-04-22T07:04:04.859993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:08.934970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862125+0000","last_change":"2024-04-22T07:04:03.794552+0000","last_active":"2024-04-22T07:04:04.862125+0000","last_peered":"2024-04-22T07:04:04.862125+0000","last_clean":"2024-04-22T07:04:04.862125+0000","last_became_active":"2024-04-22T07:04:03.794046+0000","last_became_peered":"2024-04-22T07:04:03.794046+0000","last_unstale":"2024-04-22T07:04:04.862125+0000","last_undegraded":"2024-04-22T07:04:04.862125+0000","last_fullsized":"2024-04-22T07:04:04.862125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:57:54.816996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860703+0000","last_change":"2024-04-22T07:04:03.798686+0000","last_active":"2024-04-22T07:04:04.860703+0000","last_peered":"2024-04-22T07:04:04.860703+0000","last_clean":"2024-04-22T07:04:04.860703+0000","last_became_active":"2024-04-22T07:04:03.798104+0000","last_became_peered":"2024-04-22T07:04:03.798104+0000","last_unstale":"2024-04-22T07:04:04.860703+0000","last_undegraded":"2024-04-22T07:04:04.860703+0000","last_fullsized":"2024-04-22T07:04:04.860703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860494+0000","last_change":"2024-04-22T07:04:03.794215+0000","last_active":"2024-04-22T07:04:04.860494+0000","last_peered":"2024-04-22T07:04:04.860494+0000","last_clean":"2024-04-22T07:04:04.860494+0000","last_became_active":"2024-04-22T07:04:03.793990+0000","last_became_peered":"2024-04-22T07:04:03.793990+0000","last_unstale":"2024-04-22T07:04:04.860494+0000","last_undegraded":"2024-04-22T07:04:04.860494+0000","last_fullsized":"2024-04-22T07:04:04.860494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861162+0000","last_change":"2024-04-22T07:04:03.798539+0000","last_active":"2024-04-22T07:04:04.861162+0000","last_peered":"2024-04-22T07:04:04.861162+0000","last_clean":"2024-04-22T07:04:04.861162+0000","last_became_active":"2024-04-22T07:04:03.797713+0000","last_became_peered":"2024-04-22T07:04:03.797713+0000","last_unstale":"2024-04-22T07:04:04.861162+0000","last_undegraded":"2024-04-22T07:04:04.861162+0000","last_fullsized":"2024-04-22T07:04:04.861162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:13.546338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859945+0000","last_change":"2024-04-22T07:04:03.798610+0000","last_active":"2024-04-22T07:04:04.859945+0000","last_peered":"2024-04-22T07:04:04.859945+0000","last_clean":"2024-04-22T07:04:04.859945+0000","last_became_active":"2024-04-22T07:04:03.797981+0000","last_became_peered":"2024-04-22T07:04:03.797981+0000","last_unstale":"2024-04-22T07:04:04.859945+0000","last_undegraded":"2024-04-22T07:04:04.859945+0000","last_fullsized":"2024-04-22T07:04:04.859945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861507+0000","last_change":"2024-04-22T07:04:03.802742+0000","last_active":"2024-04-22T07:04:04.861507+0000","last_peered":"2024-04-22T07:04:04.861507+0000","last_clean":"2024-04-22T07:04:04.861507+0000","last_became_active":"2024-04-22T07:04:03.802127+0000","last_became_peered":"2024-04-22T07:04:03.802127+0000","last_unstale":"2024-04-22T07:04:04.861507+0000","last_undegraded":"2024-04-22T07:04:04.861507+0000","last_fullsized":"2024-04-22T07:04:04.861507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860138+0000","last_change":"2024-04-22T07:04:03.798789+0000","last_active":"2024-04-22T07:04:04.860138+0000","last_peered":"2024-04-22T07:04:04.860138+0000","last_clean":"2024-04-22T07:04:04.860138+0000","last_became_active":"2024-04-22T07:04:03.798505+0000","last_became_peered":"2024-04-22T07:04:03.798505+0000","last_unstale":"2024-04-22T07:04:04.860138+0000","last_undegraded":"2024-04-22T07:04:04.860138+0000","last_fullsized":"2024-04-22T07:04:04.860138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860919+0000","last_change":"2024-04-22T07:04:03.794111+0000","last_active":"2024-04-22T07:04:04.860919+0000","last_peered":"2024-04-22T07:04:04.860919+0000","last_clean":"2024-04-22T07:04:04.860919+0000","last_became_active":"2024-04-22T07:04:03.793682+0000","last_became_peered":"2024-04-22T07:04:03.793682+0000","last_unstale":"2024-04-22T07:04:04.860919+0000","last_undegraded":"2024-04-22T07:04:04.860919+0000","last_fullsized":"2024-04-22T07:04:04.860919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788083+0000","last_change":"2024-04-22T07:04:03.790167+0000","last_active":"2024-04-22T07:04:04.788083+0000","last_peered":"2024-04-22T07:04:04.788083+0000","last_clean":"2024-04-22T07:04:04.788083+0000","last_became_active":"2024-04-22T07:04:03.789873+0000","last_became_peered":"2024-04-22T07:04:03.789873+0000","last_unstale":"2024-04-22T07:04:04.788083+0000","last_undegraded":"2024-04-22T07:04:04.788083+0000","last_fullsized":"2024-04-22T07:04:04.788083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859954+0000","last_change":"2024-04-22T07:04:03.798713+0000","last_active":"2024-04-22T07:04:04.859954+0000","last_peered":"2024-04-22T07:04:04.859954+0000","last_clean":"2024-04-22T07:04:04.859954+0000","last_became_active":"2024-04-22T07:04:03.798344+0000","last_became_peered":"2024-04-22T07:04:03.798344+0000","last_unstale":"2024-04-22T07:04:04.859954+0000","last_undegraded":"2024-04-22T07:04:04.859954+0000","last_fullsized":"2024-04-22T07:04:04.859954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:53.219018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860634+0000","last_change":"2024-04-22T07:04:03.795452+0000","last_active":"2024-04-22T07:04:04.860634+0000","last_peered":"2024-04-22T07:04:04.860634+0000","last_clean":"2024-04-22T07:04:04.860634+0000","last_became_active":"2024-04-22T07:04:03.794970+0000","last_became_peered":"2024-04-22T07:04:03.794970+0000","last_unstale":"2024-04-22T07:04:04.860634+0000","last_undegraded":"2024-04-22T07:04:04.860634+0000","last_fullsized":"2024-04-22T07:04:04.860634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861203+0000","last_change":"2024-04-22T07:04:03.799581+0000","last_active":"2024-04-22T07:04:04.861203+0000","last_peered":"2024-04-22T07:04:04.861203+0000","last_clean":"2024-04-22T07:04:04.861203+0000","last_became_active":"2024-04-22T07:04:03.799328+0000","last_became_peered":"2024-04-22T07:04:03.799328+0000","last_unstale":"2024-04-22T07:04:04.861203+0000","last_undegraded":"2024-04-22T07:04:04.861203+0000","last_fullsized":"2024-04-22T07:04:04.861203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859944+0000","last_change":"2024-04-22T07:04:03.796830+0000","last_active":"2024-04-22T07:04:04.859944+0000","last_peered":"2024-04-22T07:04:04.859944+0000","last_clean":"2024-04-22T07:04:04.859944+0000","last_became_active":"2024-04-22T07:04:03.796551+0000","last_became_peered":"2024-04-22T07:04:03.796551+0000","last_unstale":"2024-04-22T07:04:04.859944+0000","last_undegraded":"2024-04-22T07:04:04.859944+0000","last_fullsized":"2024-04-22T07:04:04.859944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860101+0000","last_change":"2024-04-22T07:04:03.795320+0000","last_active":"2024-04-22T07:04:04.860101+0000","last_peered":"2024-04-22T07:04:04.860101+0000","last_clean":"2024-04-22T07:04:04.860101+0000","last_became_active":"2024-04-22T07:04:03.795025+0000","last_became_peered":"2024-04-22T07:04:03.795025+0000","last_unstale":"2024-04-22T07:04:04.860101+0000","last_undegraded":"2024-04-22T07:04:04.860101+0000","last_fullsized":"2024-04-22T07:04:04.860101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:22:06.194589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790119+0000","last_change":"2024-04-22T07:04:03.788859+0000","last_active":"2024-04-22T07:04:04.790119+0000","last_peered":"2024-04-22T07:04:04.790119+0000","last_clean":"2024-04-22T07:04:04.790119+0000","last_became_active":"2024-04-22T07:04:03.788573+0000","last_became_peered":"2024-04-22T07:04:03.788573+0000","last_unstale":"2024-04-22T07:04:04.790119+0000","last_undegraded":"2024-04-22T07:04:04.790119+0000","last_fullsized":"2024-04-22T07:04:04.790119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:36.716689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.826359+0000","last_change":"2024-04-22T07:04:02.795724+0000","last_active":"2024-04-22T07:04:13.826359+0000","last_peered":"2024-04-22T07:04:13.826359+0000","last_clean":"2024-04-22T07:04:13.826359+0000","last_became_active":"2024-04-22T07:04:02.795371+0000","last_became_peered":"2024-04-22T07:04:02.795371+0000","last_unstale":"2024-04-22T07:04:13.826359+0000","last_undegraded":"2024-04-22T07:04:13.826359+0000","last_fullsized":"2024-04-22T07:04:13.826359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860689+0000","last_change":"2024-04-22T07:04:03.787030+0000","last_active":"2024-04-22T07:04:04.860689+0000","last_peered":"2024-04-22T07:04:04.860689+0000","last_clean":"2024-04-22T07:04:04.860689+0000","last_became_active":"2024-04-22T07:04:03.786357+0000","last_became_peered":"2024-04-22T07:04:03.786357+0000","last_unstale":"2024-04-22T07:04:04.860689+0000","last_undegraded":"2024-04-22T07:04:04.860689+0000","last_fullsized":"2024-04-22T07:04:04.860689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.004655+0000","last_change":"2024-04-22T07:04:02.791429+0000","last_active":"2024-04-22T07:04:24.004655+0000","last_peered":"2024-04-22T07:04:24.004655+0000","last_clean":"2024-04-22T07:04:24.004655+0000","last_became_active":"2024-04-22T07:04:02.791114+0000","last_became_peered":"2024-04-22T07:04:02.791114+0000","last_unstale":"2024-04-22T07:04:24.004655+0000","last_undegraded":"2024-04-22T07:04:24.004655+0000","last_fullsized":"2024-04-22T07:04:24.004655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.917309+0000","last_change":"2024-04-22T07:04:02.789531+0000","last_active":"2024-04-22T07:04:18.917309+0000","last_peered":"2024-04-22T07:04:18.917309+0000","last_clean":"2024-04-22T07:04:18.917309+0000","last_became_active":"2024-04-22T07:04:02.789248+0000","last_became_peered":"2024-04-22T07:04:02.789248+0000","last_unstale":"2024-04-22T07:04:18.917309+0000","last_undegraded":"2024-04-22T07:04:18.917309+0000","last_fullsized":"2024-04-22T07:04:18.917309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:06:43.031782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859624+0000","last_change":"2024-04-22T07:04:03.790142+0000","last_active":"2024-04-22T07:04:04.859624+0000","last_peered":"2024-04-22T07:04:04.859624+0000","last_clean":"2024-04-22T07:04:04.859624+0000","last_became_active":"2024-04-22T07:04:03.789869+0000","last_became_peered":"2024-04-22T07:04:03.789869+0000","last_unstale":"2024-04-22T07:04:04.859624+0000","last_undegraded":"2024-04-22T07:04:04.859624+0000","last_fullsized":"2024-04-22T07:04:04.859624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:00:09.096289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.002089+0000","last_change":"2024-04-22T07:04:02.790537+0000","last_active":"2024-04-22T07:04:24.002089+0000","last_peered":"2024-04-22T07:04:24.002089+0000","last_clean":"2024-04-22T07:04:24.002089+0000","last_became_active":"2024-04-22T07:04:02.789072+0000","last_became_peered":"2024-04-22T07:04:02.789072+0000","last_unstale":"2024-04-22T07:04:24.002089+0000","last_undegraded":"2024-04-22T07:04:24.002089+0000","last_fullsized":"2024-04-22T07:04:24.002089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787835+0000","last_change":"2024-04-22T07:04:03.791239+0000","last_active":"2024-04-22T07:04:04.787835+0000","last_peered":"2024-04-22T07:04:04.787835+0000","last_clean":"2024-04-22T07:04:04.787835+0000","last_became_active":"2024-04-22T07:04:03.790515+0000","last_became_peered":"2024-04-22T07:04:03.790515+0000","last_unstale":"2024-04-22T07:04:04.787835+0000","last_undegraded":"2024-04-22T07:04:04.787835+0000","last_fullsized":"2024-04-22T07:04:04.787835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.825000+0000","last_change":"2024-04-22T07:04:02.793582+0000","last_active":"2024-04-22T07:04:13.825000+0000","last_peered":"2024-04-22T07:04:13.825000+0000","last_clean":"2024-04-22T07:04:13.825000+0000","last_became_active":"2024-04-22T07:04:02.793281+0000","last_became_peered":"2024-04-22T07:04:02.793281+0000","last_unstale":"2024-04-22T07:04:13.825000+0000","last_undegraded":"2024-04-22T07:04:13.825000+0000","last_fullsized":"2024-04-22T07:04:13.825000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860510+0000","last_change":"2024-04-22T07:04:03.793994+0000","last_active":"2024-04-22T07:04:04.860510+0000","last_peered":"2024-04-22T07:04:04.860510+0000","last_clean":"2024-04-22T07:04:04.860510+0000","last_became_active":"2024-04-22T07:04:03.793734+0000","last_became_peered":"2024-04-22T07:04:03.793734+0000","last_unstale":"2024-04-22T07:04:04.860510+0000","last_undegraded":"2024-04-22T07:04:04.860510+0000","last_fullsized":"2024-04-22T07:04:04.860510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.034436+0000","last_change":"2024-04-22T07:04:02.791945+0000","last_active":"2024-04-22T07:04:30.034436+0000","last_peered":"2024-04-22T07:04:30.034436+0000","last_clean":"2024-04-22T07:04:30.034436+0000","last_became_active":"2024-04-22T07:04:02.791654+0000","last_became_peered":"2024-04-22T07:04:02.791654+0000","last_unstale":"2024-04-22T07:04:30.034436+0000","last_undegraded":"2024-04-22T07:04:30.034436+0000","last_fullsized":"2024-04-22T07:04:30.034436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860230+0000","last_change":"2024-04-22T07:04:03.797391+0000","last_active":"2024-04-22T07:04:04.860230+0000","last_peered":"2024-04-22T07:04:04.860230+0000","last_clean":"2024-04-22T07:04:04.860230+0000","last_became_active":"2024-04-22T07:04:03.797123+0000","last_became_peered":"2024-04-22T07:04:03.797123+0000","last_unstale":"2024-04-22T07:04:04.860230+0000","last_undegraded":"2024-04-22T07:04:04.860230+0000","last_fullsized":"2024-04-22T07:04:04.860230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.833517+0000","last_change":"2024-04-22T07:04:02.793753+0000","last_active":"2024-04-22T07:04:13.833517+0000","last_peered":"2024-04-22T07:04:13.833517+0000","last_clean":"2024-04-22T07:04:13.833517+0000","last_became_active":"2024-04-22T07:04:02.793436+0000","last_became_peered":"2024-04-22T07:04:02.793436+0000","last_unstale":"2024-04-22T07:04:13.833517+0000","last_undegraded":"2024-04-22T07:04:13.833517+0000","last_fullsized":"2024-04-22T07:04:13.833517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790527+0000","last_change":"2024-04-22T07:04:03.800665+0000","last_active":"2024-04-22T07:04:04.790527+0000","last_peered":"2024-04-22T07:04:04.790527+0000","last_clean":"2024-04-22T07:04:04.790527+0000","last_became_active":"2024-04-22T07:04:03.800227+0000","last_became_peered":"2024-04-22T07:04:03.800227+0000","last_unstale":"2024-04-22T07:04:04.790527+0000","last_undegraded":"2024-04-22T07:04:04.790527+0000","last_fullsized":"2024-04-22T07:04:04.790527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:43.296063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'10","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.031633+0000","last_change":"2024-04-22T07:04:02.791309+0000","last_active":"2024-04-22T07:04:30.031633+0000","last_peered":"2024-04-22T07:04:30.031633+0000","last_clean":"2024-04-22T07:04:30.031633+0000","last_became_active":"2024-04-22T07:04:02.791000+0000","last_became_peered":"2024-04-22T07:04:02.791000+0000","last_unstale":"2024-04-22T07:04:30.031633+0000","last_undegraded":"2024-04-22T07:04:30.031633+0000","last_fullsized":"2024-04-22T07:04:30.031633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861543+0000","last_change":"2024-04-22T07:04:03.794316+0000","last_active":"2024-04-22T07:04:04.861543+0000","last_peered":"2024-04-22T07:04:04.861543+0000","last_clean":"2024-04-22T07:04:04.861543+0000","last_became_active":"2024-04-22T07:04:03.793646+0000","last_became_peered":"2024-04-22T07:04:03.793646+0000","last_unstale":"2024-04-22T07:04:04.861543+0000","last_undegraded":"2024-04-22T07:04:04.861543+0000","last_fullsized":"2024-04-22T07:04:04.861543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:25:27.378772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.033424+0000","last_change":"2024-04-22T07:04:02.791024+0000","last_active":"2024-04-22T07:04:30.033424+0000","last_peered":"2024-04-22T07:04:30.033424+0000","last_clean":"2024-04-22T07:04:30.033424+0000","last_became_active":"2024-04-22T07:04:02.789160+0000","last_became_peered":"2024-04-22T07:04:02.789160+0000","last_unstale":"2024-04-22T07:04:30.033424+0000","last_undegraded":"2024-04-22T07:04:30.033424+0000","last_fullsized":"2024-04-22T07:04:30.033424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:07.496976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859792+0000","last_change":"2024-04-22T07:04:03.798436+0000","last_active":"2024-04-22T07:04:04.859792+0000","last_peered":"2024-04-22T07:04:04.859792+0000","last_clean":"2024-04-22T07:04:04.859792+0000","last_became_active":"2024-04-22T07:04:03.797946+0000","last_became_peered":"2024-04-22T07:04:03.797946+0000","last_unstale":"2024-04-22T07:04:04.859792+0000","last_undegraded":"2024-04-22T07:04:04.859792+0000","last_fullsized":"2024-04-22T07:04:04.859792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.001390+0000","last_change":"2024-04-22T07:04:02.796551+0000","last_active":"2024-04-22T07:04:24.001390+0000","last_peered":"2024-04-22T07:04:24.001390+0000","last_clean":"2024-04-22T07:04:24.001390+0000","last_became_active":"2024-04-22T07:04:02.796332+0000","last_became_peered":"2024-04-22T07:04:02.796332+0000","last_unstale":"2024-04-22T07:04:24.001390+0000","last_undegraded":"2024-04-22T07:04:24.001390+0000","last_fullsized":"2024-04-22T07:04:24.001390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:15.854295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861758+0000","last_change":"2024-04-22T07:04:03.804622+0000","last_active":"2024-04-22T07:04:04.861758+0000","last_peered":"2024-04-22T07:04:04.861758+0000","last_clean":"2024-04-22T07:04:04.861758+0000","last_became_active":"2024-04-22T07:04:03.804324+0000","last_became_peered":"2024-04-22T07:04:03.804324+0000","last_unstale":"2024-04-22T07:04:04.861758+0000","last_undegraded":"2024-04-22T07:04:04.861758+0000","last_fullsized":"2024-04-22T07:04:04.861758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:21:24.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.002064+0000","last_change":"2024-04-22T07:04:02.787085+0000","last_active":"2024-04-22T07:04:24.002064+0000","last_peered":"2024-04-22T07:04:24.002064+0000","last_clean":"2024-04-22T07:04:24.002064+0000","last_became_active":"2024-04-22T07:04:02.786777+0000","last_became_peered":"2024-04-22T07:04:02.786777+0000","last_unstale":"2024-04-22T07:04:24.002064+0000","last_undegraded":"2024-04-22T07:04:24.002064+0000","last_fullsized":"2024-04-22T07:04:24.002064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860535+0000","last_change":"2024-04-22T07:04:03.796356+0000","last_active":"2024-04-22T07:04:04.860535+0000","last_peered":"2024-04-22T07:04:04.860535+0000","last_clean":"2024-04-22T07:04:04.860535+0000","last_became_active":"2024-04-22T07:04:03.796030+0000","last_became_peered":"2024-04-22T07:04:03.796030+0000","last_unstale":"2024-04-22T07:04:04.860535+0000","last_undegraded":"2024-04-22T07:04:04.860535+0000","last_fullsized":"2024-04-22T07:04:04.860535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.000472+0000","last_change":"2024-04-22T07:04:02.787425+0000","last_active":"2024-04-22T07:04:24.000472+0000","last_peered":"2024-04-22T07:04:24.000472+0000","last_clean":"2024-04-22T07:04:24.000472+0000","last_became_active":"2024-04-22T07:04:02.787146+0000","last_became_peered":"2024-04-22T07:04:02.787146+0000","last_unstale":"2024-04-22T07:04:24.000472+0000","last_undegraded":"2024-04-22T07:04:24.000472+0000","last_fullsized":"2024-04-22T07:04:24.000472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:54:46.552132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861631+0000","last_change":"2024-04-22T07:04:03.802924+0000","last_active":"2024-04-22T07:04:04.861631+0000","last_peered":"2024-04-22T07:04:04.861631+0000","last_clean":"2024-04-22T07:04:04.861631+0000","last_became_active":"2024-04-22T07:04:03.802644+0000","last_became_peered":"2024-04-22T07:04:03.802644+0000","last_unstale":"2024-04-22T07:04:04.861631+0000","last_undegraded":"2024-04-22T07:04:04.861631+0000","last_fullsized":"2024-04-22T07:04:04.861631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:10.925597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.783163+0000","last_change":"2024-04-22T07:04:02.794481+0000","last_active":"2024-04-22T07:04:08.783163+0000","last_peered":"2024-04-22T07:04:08.783163+0000","last_clean":"2024-04-22T07:04:08.783163+0000","last_became_active":"2024-04-22T07:04:02.793982+0000","last_became_peered":"2024-04-22T07:04:02.793982+0000","last_unstale":"2024-04-22T07:04:08.783163+0000","last_undegraded":"2024-04-22T07:04:08.783163+0000","last_fullsized":"2024-04-22T07:04:08.783163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:58.906416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862139+0000","last_change":"2024-04-22T07:04:03.802724+0000","last_active":"2024-04-22T07:04:04.862139+0000","last_peered":"2024-04-22T07:04:04.862139+0000","last_clean":"2024-04-22T07:04:04.862139+0000","last_became_active":"2024-04-22T07:04:03.802434+0000","last_became_peered":"2024-04-22T07:04:03.802434+0000","last_unstale":"2024-04-22T07:04:04.862139+0000","last_undegraded":"2024-04-22T07:04:04.862139+0000","last_fullsized":"2024-04-22T07:04:04.862139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.030841+0000","last_change":"2024-04-22T07:04:02.791375+0000","last_active":"2024-04-22T07:04:30.030841+0000","last_peered":"2024-04-22T07:04:30.030841+0000","last_clean":"2024-04-22T07:04:30.030841+0000","last_became_active":"2024-04-22T07:04:02.791095+0000","last_became_peered":"2024-04-22T07:04:02.791095+0000","last_unstale":"2024-04-22T07:04:30.030841+0000","last_undegraded":"2024-04-22T07:04:30.030841+0000","last_fullsized":"2024-04-22T07:04:30.030841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862265+0000","last_change":"2024-04-22T07:04:03.802028+0000","last_active":"2024-04-22T07:04:04.862265+0000","last_peered":"2024-04-22T07:04:04.862265+0000","last_clean":"2024-04-22T07:04:04.862265+0000","last_became_active":"2024-04-22T07:04:03.801715+0000","last_became_peered":"2024-04-22T07:04:03.801715+0000","last_unstale":"2024-04-22T07:04:04.862265+0000","last_undegraded":"2024-04-22T07:04:04.862265+0000","last_fullsized":"2024-04-22T07:04:04.862265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:29.815068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.002037+0000","last_change":"2024-04-22T07:04:02.793270+0000","last_active":"2024-04-22T07:04:24.002037+0000","last_peered":"2024-04-22T07:04:24.002037+0000","last_clean":"2024-04-22T07:04:24.002037+0000","last_became_active":"2024-04-22T07:04:02.793009+0000","last_became_peered":"2024-04-22T07:04:02.793009+0000","last_unstale":"2024-04-22T07:04:24.002037+0000","last_undegraded":"2024-04-22T07:04:24.002037+0000","last_fullsized":"2024-04-22T07:04:24.002037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861459+0000","last_change":"2024-04-22T07:04:03.800175+0000","last_active":"2024-04-22T07:04:04.861459+0000","last_peered":"2024-04-22T07:04:04.861459+0000","last_clean":"2024-04-22T07:04:04.861459+0000","last_became_active":"2024-04-22T07:04:03.799902+0000","last_became_peered":"2024-04-22T07:04:03.799902+0000","last_unstale":"2024-04-22T07:04:04.861459+0000","last_undegraded":"2024-04-22T07:04:04.861459+0000","last_fullsized":"2024-04-22T07:04:04.861459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:34:45.024040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.001001+0000","last_change":"2024-04-22T07:04:02.793643+0000","last_active":"2024-04-22T07:04:24.001001+0000","last_peered":"2024-04-22T07:04:24.001001+0000","last_clean":"2024-04-22T07:04:24.001001+0000","last_became_active":"2024-04-22T07:04:02.793333+0000","last_became_peered":"2024-04-22T07:04:02.793333+0000","last_unstale":"2024-04-22T07:04:24.001001+0000","last_undegraded":"2024-04-22T07:04:24.001001+0000","last_fullsized":"2024-04-22T07:04:24.001001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:22:46.012964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.862019+0000","last_change":"2024-04-22T07:04:03.801821+0000","last_active":"2024-04-22T07:04:04.862019+0000","last_peered":"2024-04-22T07:04:04.862019+0000","last_clean":"2024-04-22T07:04:04.862019+0000","last_became_active":"2024-04-22T07:04:03.801510+0000","last_became_peered":"2024-04-22T07:04:03.801510+0000","last_unstale":"2024-04-22T07:04:04.862019+0000","last_undegraded":"2024-04-22T07:04:04.862019+0000","last_fullsized":"2024-04-22T07:04:04.862019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861241+0000","last_change":"2024-04-22T07:03:59.215285+0000","last_active":"2024-04-22T07:04:04.861241+0000","last_peered":"2024-04-22T07:04:04.861241+0000","last_clean":"2024-04-22T07:04:04.861241+0000","last_became_active":"2024-04-22T07:03:59.214275+0000","last_became_peered":"2024-04-22T07:03:59.214275+0000","last_unstale":"2024-04-22T07:04:04.861241+0000","last_undegraded":"2024-04-22T07:04:04.861241+0000","last_fullsized":"2024-04-22T07:04:04.861241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:03:57.959761+0000","last_clean_scrub_stamp":"2024-04-22T07:03:57.959761+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:55.105073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.000978+0000","last_change":"2024-04-22T07:04:02.793384+0000","last_active":"2024-04-22T07:04:24.000978+0000","last_peered":"2024-04-22T07:04:24.000978+0000","last_clean":"2024-04-22T07:04:24.000978+0000","last_became_active":"2024-04-22T07:04:02.793077+0000","last_became_peered":"2024-04-22T07:04:02.793077+0000","last_unstale":"2024-04-22T07:04:24.000978+0000","last_undegraded":"2024-04-22T07:04:24.000978+0000","last_fullsized":"2024-04-22T07:04:24.000978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860779+0000","last_change":"2024-04-22T07:04:03.798134+0000","last_active":"2024-04-22T07:04:04.860779+0000","last_peered":"2024-04-22T07:04:04.860779+0000","last_clean":"2024-04-22T07:04:04.860779+0000","last_became_active":"2024-04-22T07:04:03.797848+0000","last_became_peered":"2024-04-22T07:04:03.797848+0000","last_unstale":"2024-04-22T07:04:04.860779+0000","last_undegraded":"2024-04-22T07:04:04.860779+0000","last_fullsized":"2024-04-22T07:04:04.860779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:33:38.291479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.915772+0000","last_change":"2024-04-22T07:04:02.789591+0000","last_active":"2024-04-22T07:04:18.915772+0000","last_peered":"2024-04-22T07:04:18.915772+0000","last_clean":"2024-04-22T07:04:18.915772+0000","last_became_active":"2024-04-22T07:04:02.788785+0000","last_became_peered":"2024-04-22T07:04:02.788785+0000","last_unstale":"2024-04-22T07:04:18.915772+0000","last_undegraded":"2024-04-22T07:04:18.915772+0000","last_fullsized":"2024-04-22T07:04:18.915772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859611+0000","last_change":"2024-04-22T07:04:03.789757+0000","last_active":"2024-04-22T07:04:04.859611+0000","last_peered":"2024-04-22T07:04:04.859611+0000","last_clean":"2024-04-22T07:04:04.859611+0000","last_became_active":"2024-04-22T07:04:03.789463+0000","last_became_peered":"2024-04-22T07:04:03.789463+0000","last_unstale":"2024-04-22T07:04:04.859611+0000","last_undegraded":"2024-04-22T07:04:04.859611+0000","last_fullsized":"2024-04-22T07:04:04.859611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:48:16.859332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.915436+0000","last_change":"2024-04-22T07:04:02.790600+0000","last_active":"2024-04-22T07:04:18.915436+0000","last_peered":"2024-04-22T07:04:18.915436+0000","last_clean":"2024-04-22T07:04:18.915436+0000","last_became_active":"2024-04-22T07:04:02.790319+0000","last_became_peered":"2024-04-22T07:04:02.790319+0000","last_unstale":"2024-04-22T07:04:18.915436+0000","last_undegraded":"2024-04-22T07:04:18.915436+0000","last_fullsized":"2024-04-22T07:04:18.915436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:36.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787840+0000","last_change":"2024-04-22T07:04:03.788231+0000","last_active":"2024-04-22T07:04:04.787840+0000","last_peered":"2024-04-22T07:04:04.787840+0000","last_clean":"2024-04-22T07:04:04.787840+0000","last_became_active":"2024-04-22T07:04:03.787918+0000","last_became_peered":"2024-04-22T07:04:03.787918+0000","last_unstale":"2024-04-22T07:04:04.787840+0000","last_undegraded":"2024-04-22T07:04:04.787840+0000","last_fullsized":"2024-04-22T07:04:04.787840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:39.746369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.036874+0000","last_change":"2024-04-22T07:04:02.796245+0000","last_active":"2024-04-22T07:04:30.036874+0000","last_peered":"2024-04-22T07:04:30.036874+0000","last_clean":"2024-04-22T07:04:30.036874+0000","last_became_active":"2024-04-22T07:04:02.795947+0000","last_became_peered":"2024-04-22T07:04:02.795947+0000","last_unstale":"2024-04-22T07:04:30.036874+0000","last_undegraded":"2024-04-22T07:04:30.036874+0000","last_fullsized":"2024-04-22T07:04:30.036874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790274+0000","last_change":"2024-04-22T07:04:03.793330+0000","last_active":"2024-04-22T07:04:04.790274+0000","last_peered":"2024-04-22T07:04:04.790274+0000","last_clean":"2024-04-22T07:04:04.790274+0000","last_became_active":"2024-04-22T07:04:03.793037+0000","last_became_peered":"2024-04-22T07:04:03.793037+0000","last_unstale":"2024-04-22T07:04:04.790274+0000","last_undegraded":"2024-04-22T07:04:04.790274+0000","last_fullsized":"2024-04-22T07:04:04.790274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:13.368853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.787998+0000","last_change":"2024-04-22T07:04:02.793498+0000","last_active":"2024-04-22T07:04:04.787998+0000","last_peered":"2024-04-22T07:04:04.787998+0000","last_clean":"2024-04-22T07:04:04.787998+0000","last_became_active":"2024-04-22T07:04:02.793215+0000","last_became_peered":"2024-04-22T07:04:02.793215+0000","last_unstale":"2024-04-22T07:04:04.787998+0000","last_undegraded":"2024-04-22T07:04:04.787998+0000","last_fullsized":"2024-04-22T07:04:04.787998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860866+0000","last_change":"2024-04-22T07:04:03.804152+0000","last_active":"2024-04-22T07:04:04.860866+0000","last_peered":"2024-04-22T07:04:04.860866+0000","last_clean":"2024-04-22T07:04:04.860866+0000","last_became_active":"2024-04-22T07:04:03.801961+0000","last_became_peered":"2024-04-22T07:04:03.801961+0000","last_unstale":"2024-04-22T07:04:04.860866+0000","last_undegraded":"2024-04-22T07:04:04.860866+0000","last_fullsized":"2024-04-22T07:04:04.860866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.004118+0000","last_change":"2024-04-22T07:04:02.793445+0000","last_active":"2024-04-22T07:04:24.004118+0000","last_peered":"2024-04-22T07:04:24.004118+0000","last_clean":"2024-04-22T07:04:24.004118+0000","last_became_active":"2024-04-22T07:04:02.793164+0000","last_became_peered":"2024-04-22T07:04:02.793164+0000","last_unstale":"2024-04-22T07:04:24.004118+0000","last_undegraded":"2024-04-22T07:04:24.004118+0000","last_fullsized":"2024-04-22T07:04:24.004118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.788013+0000","last_change":"2024-04-22T07:04:03.789589+0000","last_active":"2024-04-22T07:04:04.788013+0000","last_peered":"2024-04-22T07:04:04.788013+0000","last_clean":"2024-04-22T07:04:04.788013+0000","last_became_active":"2024-04-22T07:04:03.789310+0000","last_became_peered":"2024-04-22T07:04:03.789310+0000","last_unstale":"2024-04-22T07:04:04.788013+0000","last_undegraded":"2024-04-22T07:04:04.788013+0000","last_fullsized":"2024-04-22T07:04:04.788013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.004469+0000","last_change":"2024-04-22T07:04:02.789243+0000","last_active":"2024-04-22T07:04:24.004469+0000","last_peered":"2024-04-22T07:04:24.004469+0000","last_clean":"2024-04-22T07:04:24.004469+0000","last_became_active":"2024-04-22T07:04:02.788897+0000","last_became_peered":"2024-04-22T07:04:02.788897+0000","last_unstale":"2024-04-22T07:04:24.004469+0000","last_undegraded":"2024-04-22T07:04:24.004469+0000","last_fullsized":"2024-04-22T07:04:24.004469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:07.929933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860272+0000","last_change":"2024-04-22T07:04:03.793656+0000","last_active":"2024-04-22T07:04:04.860272+0000","last_peered":"2024-04-22T07:04:04.860272+0000","last_clean":"2024-04-22T07:04:04.860272+0000","last_became_active":"2024-04-22T07:04:03.793311+0000","last_became_peered":"2024-04-22T07:04:03.793311+0000","last_unstale":"2024-04-22T07:04:04.860272+0000","last_undegraded":"2024-04-22T07:04:04.860272+0000","last_fullsized":"2024-04-22T07:04:04.860272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:30.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.032054+0000","last_change":"2024-04-22T07:04:02.786878+0000","last_active":"2024-04-22T07:04:30.032054+0000","last_peered":"2024-04-22T07:04:30.032054+0000","last_clean":"2024-04-22T07:04:30.032054+0000","last_became_active":"2024-04-22T07:04:02.786526+0000","last_became_peered":"2024-04-22T07:04:02.786526+0000","last_unstale":"2024-04-22T07:04:30.032054+0000","last_undegraded":"2024-04-22T07:04:30.032054+0000","last_fullsized":"2024-04-22T07:04:30.032054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:30:35.711652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860520+0000","last_change":"2024-04-22T07:04:03.791544+0000","last_active":"2024-04-22T07:04:04.860520+0000","last_peered":"2024-04-22T07:04:04.860520+0000","last_clean":"2024-04-22T07:04:04.860520+0000","last_became_active":"2024-04-22T07:04:03.791289+0000","last_became_peered":"2024-04-22T07:04:03.791289+0000","last_unstale":"2024-04-22T07:04:04.860520+0000","last_undegraded":"2024-04-22T07:04:04.860520+0000","last_fullsized":"2024-04-22T07:04:04.860520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:03:14.805608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.824240+0000","last_change":"2024-04-22T07:04:02.795178+0000","last_active":"2024-04-22T07:04:13.824240+0000","last_peered":"2024-04-22T07:04:13.824240+0000","last_clean":"2024-04-22T07:04:13.824240+0000","last_became_active":"2024-04-22T07:04:02.794795+0000","last_became_peered":"2024-04-22T07:04:02.794795+0000","last_unstale":"2024-04-22T07:04:13.824240+0000","last_undegraded":"2024-04-22T07:04:13.824240+0000","last_fullsized":"2024-04-22T07:04:13.824240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.790163+0000","last_change":"2024-04-22T07:04:03.788560+0000","last_active":"2024-04-22T07:04:04.790163+0000","last_peered":"2024-04-22T07:04:04.790163+0000","last_clean":"2024-04-22T07:04:04.790163+0000","last_became_active":"2024-04-22T07:04:03.788278+0000","last_became_peered":"2024-04-22T07:04:03.788278+0000","last_unstale":"2024-04-22T07:04:04.790163+0000","last_undegraded":"2024-04-22T07:04:04.790163+0000","last_fullsized":"2024-04-22T07:04:04.790163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:41:37.492038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.034048+0000","last_change":"2024-04-22T07:04:02.788532+0000","last_active":"2024-04-22T07:04:30.034048+0000","last_peered":"2024-04-22T07:04:30.034048+0000","last_clean":"2024-04-22T07:04:30.034048+0000","last_became_active":"2024-04-22T07:04:02.788254+0000","last_became_peered":"2024-04-22T07:04:02.788254+0000","last_unstale":"2024-04-22T07:04:30.034048+0000","last_undegraded":"2024-04-22T07:04:30.034048+0000","last_fullsized":"2024-04-22T07:04:30.034048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:27:42.291026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.859748+0000","last_change":"2024-04-22T07:04:03.790052+0000","last_active":"2024-04-22T07:04:04.859748+0000","last_peered":"2024-04-22T07:04:04.859748+0000","last_clean":"2024-04-22T07:04:04.859748+0000","last_became_active":"2024-04-22T07:04:03.789351+0000","last_became_peered":"2024-04-22T07:04:03.789351+0000","last_unstale":"2024-04-22T07:04:04.859748+0000","last_undegraded":"2024-04-22T07:04:04.859748+0000","last_fullsized":"2024-04-22T07:04:04.859748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:24.000434+0000","last_change":"2024-04-22T07:04:02.792774+0000","last_active":"2024-04-22T07:04:24.000434+0000","last_peered":"2024-04-22T07:04:24.000434+0000","last_clean":"2024-04-22T07:04:24.000434+0000","last_became_active":"2024-04-22T07:04:02.792282+0000","last_became_peered":"2024-04-22T07:04:02.792282+0000","last_unstale":"2024-04-22T07:04:24.000434+0000","last_undegraded":"2024-04-22T07:04:24.000434+0000","last_fullsized":"2024-04-22T07:04:24.000434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:16.282031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860691+0000","last_change":"2024-04-22T07:04:03.796133+0000","last_active":"2024-04-22T07:04:04.860691+0000","last_peered":"2024-04-22T07:04:04.860691+0000","last_clean":"2024-04-22T07:04:04.860691+0000","last_became_active":"2024-04-22T07:04:03.795854+0000","last_became_peered":"2024-04-22T07:04:03.795854+0000","last_unstale":"2024-04-22T07:04:04.860691+0000","last_undegraded":"2024-04-22T07:04:04.860691+0000","last_fullsized":"2024-04-22T07:04:04.860691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:05:46.154561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:13.822027+0000","last_change":"2024-04-22T07:04:02.789099+0000","last_active":"2024-04-22T07:04:13.822027+0000","last_peered":"2024-04-22T07:04:13.822027+0000","last_clean":"2024-04-22T07:04:13.822027+0000","last_became_active":"2024-04-22T07:04:02.788782+0000","last_became_peered":"2024-04-22T07:04:02.788782+0000","last_unstale":"2024-04-22T07:04:13.822027+0000","last_undegraded":"2024-04-22T07:04:13.822027+0000","last_fullsized":"2024-04-22T07:04:13.822027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:45:32.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860112+0000","last_change":"2024-04-22T07:04:03.799845+0000","last_active":"2024-04-22T07:04:04.860112+0000","last_peered":"2024-04-22T07:04:04.860112+0000","last_clean":"2024-04-22T07:04:04.860112+0000","last_became_active":"2024-04-22T07:04:03.799553+0000","last_became_peered":"2024-04-22T07:04:03.799553+0000","last_unstale":"2024-04-22T07:04:04.860112+0000","last_undegraded":"2024-04-22T07:04:04.860112+0000","last_fullsized":"2024-04-22T07:04:04.860112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:27.492484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860699+0000","last_change":"2024-04-22T07:04:03.799918+0000","last_active":"2024-04-22T07:04:04.860699+0000","last_peered":"2024-04-22T07:04:04.860699+0000","last_clean":"2024-04-22T07:04:04.860699+0000","last_became_active":"2024-04-22T07:04:03.797851+0000","last_became_peered":"2024-04-22T07:04:03.797851+0000","last_unstale":"2024-04-22T07:04:04.860699+0000","last_undegraded":"2024-04-22T07:04:04.860699+0000","last_fullsized":"2024-04-22T07:04:04.860699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.035109+0000","last_change":"2024-04-22T07:04:02.793627+0000","last_active":"2024-04-22T07:04:30.035109+0000","last_peered":"2024-04-22T07:04:30.035109+0000","last_clean":"2024-04-22T07:04:30.035109+0000","last_became_active":"2024-04-22T07:04:02.793341+0000","last_became_peered":"2024-04-22T07:04:02.793341+0000","last_unstale":"2024-04-22T07:04:30.035109+0000","last_undegraded":"2024-04-22T07:04:30.035109+0000","last_fullsized":"2024-04-22T07:04:30.035109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:21:07.101370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.861294+0000","last_change":"2024-04-22T07:04:03.797768+0000","last_active":"2024-04-22T07:04:04.861294+0000","last_peered":"2024-04-22T07:04:04.861294+0000","last_clean":"2024-04-22T07:04:04.861294+0000","last_became_active":"2024-04-22T07:04:03.797437+0000","last_became_peered":"2024-04-22T07:04:03.797437+0000","last_unstale":"2024-04-22T07:04:04.861294+0000","last_undegraded":"2024-04-22T07:04:04.861294+0000","last_fullsized":"2024-04-22T07:04:04.861294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:56.132054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.033921+0000","last_change":"2024-04-22T07:04:02.789875+0000","last_active":"2024-04-22T07:04:30.033921+0000","last_peered":"2024-04-22T07:04:30.033921+0000","last_clean":"2024-04-22T07:04:30.033921+0000","last_became_active":"2024-04-22T07:04:02.788989+0000","last_became_peered":"2024-04-22T07:04:02.788989+0000","last_unstale":"2024-04-22T07:04:30.033921+0000","last_undegraded":"2024-04-22T07:04:30.033921+0000","last_fullsized":"2024-04-22T07:04:30.033921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:12.363816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860096+0000","last_change":"2024-04-22T07:04:03.797671+0000","last_active":"2024-04-22T07:04:04.860096+0000","last_peered":"2024-04-22T07:04:04.860096+0000","last_clean":"2024-04-22T07:04:04.860096+0000","last_became_active":"2024-04-22T07:04:03.797399+0000","last_became_peered":"2024-04-22T07:04:03.797399+0000","last_unstale":"2024-04-22T07:04:04.860096+0000","last_undegraded":"2024-04-22T07:04:04.860096+0000","last_fullsized":"2024-04-22T07:04:04.860096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:32.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:08.780760+0000","last_change":"2024-04-22T07:04:02.796141+0000","last_active":"2024-04-22T07:04:08.780760+0000","last_peered":"2024-04-22T07:04:08.780760+0000","last_clean":"2024-04-22T07:04:08.780760+0000","last_became_active":"2024-04-22T07:04:02.795821+0000","last_became_peered":"2024-04-22T07:04:02.795821+0000","last_unstale":"2024-04-22T07:04:08.780760+0000","last_undegraded":"2024-04-22T07:04:08.780760+0000","last_fullsized":"2024-04-22T07:04:08.780760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:04.860371+0000","last_change":"2024-04-22T07:04:03.796729+0000","last_active":"2024-04-22T07:04:04.860371+0000","last_peered":"2024-04-22T07:04:04.860371+0000","last_clean":"2024-04-22T07:04:04.860371+0000","last_became_active":"2024-04-22T07:04:03.796486+0000","last_became_peered":"2024-04-22T07:04:03.796486+0000","last_unstale":"2024-04-22T07:04:04.860371+0000","last_undegraded":"2024-04-22T07:04:04.860371+0000","last_fullsized":"2024-04-22T07:04:04.860371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.766141+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.766141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:47.387745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-22T07:04:30.033128+0000","last_change":"2024-04-22T07:04:02.793214+0000","last_active":"2024-04-22T07:04:30.033128+0000","last_peered":"2024-04-22T07:04:30.033128+0000","last_clean":"2024-04-22T07:04:30.033128+0000","last_became_active":"2024-04-22T07:04:02.792922+0000","last_became_peered":"2024-04-22T07:04:02.792922+0000","last_unstale":"2024-04-22T07:04:30.033128+0000","last_undegraded":"2024-04-22T07:04:30.033128+0000","last_fullsized":"2024-04-22T07:04:30.033128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:01.761104+0000","last_clean_scrub_stamp":"2024-04-22T07:04:01.761104+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:22.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":205,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":663402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":74866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":73370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":827392,"data_stored":662176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:42.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:42.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.442+0000 7fee69274640 1 Processor -- start 2024-04-22T07:04:42.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.442+0000 7fee69274640 1 -- start start 2024-04-22T07:04:42.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee69274640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fee640d0b70 0x7fee64154a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee69274640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee6415e9b0 0x7fee64154f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee69274640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee6415ab30 con 0x7fee640d0f50 2024-04-22T07:04:42.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee69274640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fee6415ba00 con 0x7fee640d0b70 2024-04-22T07:04:42.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee69274640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fee6415adf0 con 0x7fee6415e9b0 2024-04-22T07:04:42.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee62ffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fee640d0b70 0x7fee64154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee62ffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fee640d0b70 0x7fee64154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51548/0 (socket says 172.21.15.92:51548) 2024-04-22T07:04:42.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee62ffd640 1 -- 172.21.15.92:0/2271991818 learned_addr learned my addr 172.21.15.92:0/2271991818 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:42.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee637fe640 1 --2- 172.21.15.92:0/2271991818 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee6415e9b0 0x7fee64154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee62ffd640 1 -- 172.21.15.92:0/2271991818 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee6415e9b0 msgr2=0x7fee64154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee62ffd640 1 --2- 172.21.15.92:0/2271991818 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee6415e9b0 0x7fee64154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee62ffd640 1 -- 172.21.15.92:0/2271991818 >> v1:172.21.15.145:6789/0 conn(0x7fee640d0f50 legacy=0x7fee6415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee62ffd640 1 -- 172.21.15.92:0/2271991818 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6415b0d0 con 0x7fee640d0b70 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.443+0000 7fee637fe640 1 --2- 172.21.15.92:0/2271991818 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee6415e9b0 0x7fee64154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.444+0000 7fee62ffd640 1 --2- 172.21.15.92:0/2271991818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee640d0b70 0x7fee64154a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fee54009e50 tx=0x7fee5402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8aae3feb41d18a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.444+0000 7fee61ffb640 1 -- 172.21.15.92:0/2271991818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5402c9e0 con 0x7fee640d0b70 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.444+0000 7fee61ffb640 1 -- 172.21.15.92:0/2271991818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee5402cb80 con 0x7fee640d0b70 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.444+0000 7fee61ffb640 1 -- 172.21.15.92:0/2271991818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee540343f0 con 0x7fee640d0b70 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.445+0000 7fee69274640 1 -- 172.21.15.92:0/2271991818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee640d0b70 msgr2=0x7fee64154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.445+0000 7fee69274640 1 --2- 172.21.15.92:0/2271991818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee640d0b70 0x7fee64154a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fee54009e50 tx=0x7fee5402be40 comp rx=0 tx=0).stop 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.445+0000 7fee69274640 1 -- 172.21.15.92:0/2271991818 shutdown_connections 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.445+0000 7fee69274640 1 --2- 172.21.15.92:0/2271991818 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee6415e9b0 0x7fee64154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.445+0000 7fee69274640 1 --2- 172.21.15.92:0/2271991818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee640d0b70 0x7fee64154a00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.445+0000 7fee69274640 1 -- 172.21.15.92:0/2271991818 >> 172.21.15.92:0/2271991818 conn(0x7fee640736b0 msgr2=0x7fee64073ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.445+0000 7fee69274640 1 -- 172.21.15.92:0/2271991818 shutdown_connections 2024-04-22T07:04:42.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.445+0000 7fee69274640 1 -- 172.21.15.92:0/2271991818 wait complete. 2024-04-22T07:04:42.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.446+0000 7fee69274640 1 Processor -- start 2024-04-22T07:04:42.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.446+0000 7fee69274640 1 -- start start 2024-04-22T07:04:42.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee69274640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee640d0b70 0x7fee640c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee69274640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee640d0f50 0x7fee640ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee69274640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee6415e9b0 0x7fee64156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee69274640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fee640cae00 con 0x7fee6415e9b0 2024-04-22T07:04:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee69274640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fee6415b690 con 0x7fee640d0b70 2024-04-22T07:04:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee69274640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fee640caf80 con 0x7fee640d0f50 2024-04-22T07:04:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee6415e9b0 0x7fee64156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee6415e9b0 0x7fee64156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51562/0 (socket says 172.21.15.92:51562) 2024-04-22T07:04:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee637fe640 1 -- 172.21.15.92:0/2476389341 learned_addr learned my addr 172.21.15.92:0/2476389341 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee627fc640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee640d0f50 0x7fee640ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee62ffd640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee640d0b70 0x7fee640c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee637fe640 1 -- 172.21.15.92:0/2476389341 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee640d0f50 msgr2=0x7fee640ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee637fe640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee640d0f50 0x7fee640ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee637fe640 1 -- 172.21.15.92:0/2476389341 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee640d0b70 msgr2=0x7fee640c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee637fe640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee640d0b70 0x7fee640c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee637fe640 1 -- 172.21.15.92:0/2476389341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee641f0a20 con 0x7fee6415e9b0 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee62ffd640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee640d0b70 0x7fee640c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee637fe640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee6415e9b0 0x7fee64156440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00c7c0 tx=0x7fee5c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee4b7fe640 1 -- 172.21.15.92:0/2476389341 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c015020 con 0x7fee6415e9b0 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee4b7fe640 1 -- 172.21.15.92:0/2476389341 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee5c004230 con 0x7fee6415e9b0 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee69274640 1 -- 172.21.15.92:0/2476389341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee641f0d00 con 0x7fee6415e9b0 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee4b7fe640 1 -- 172.21.15.92:0/2476389341 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c007620 con 0x7fee6415e9b0 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.447+0000 7fee69274640 1 -- 172.21.15.92:0/2476389341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee641f12c0 con 0x7fee6415e9b0 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.448+0000 7fee69274640 1 -- 172.21.15.92:0/2476389341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee64069930 con 0x7fee6415e9b0 2024-04-22T07:04:42.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.449+0000 7fee4b7fe640 1 -- 172.21.15.92:0/2476389341 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee5c018020 con 0x7fee6415e9b0 2024-04-22T07:04:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.449+0000 7fee4b7fe640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fee5006c8f0 0x7fee5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.449+0000 7fee4b7fe640 1 -- 172.21.15.92:0/2476389341 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fee5c08bc00 con 0x7fee6415e9b0 2024-04-22T07:04:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.449+0000 7fee62ffd640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fee5006c8f0 0x7fee5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.450+0000 7fee62ffd640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fee5006c8f0 0x7fee5006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fee54035000 tx=0x7fee54030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.454+0000 7fee4b7fe640 1 -- 172.21.15.92:0/2476389341 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee5c092080 con 0x7fee6415e9b0 2024-04-22T07:04:42.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.666+0000 7fee69274640 1 -- 172.21.15.92:0/2476389341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee64154490 con 0x7fee6415e9b0 2024-04-22T07:04:42.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.666+0000 7fee4b7fe640 1 -- 172.21.15.92:0/2476389341 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6913 (secure 0 0 0) 0x7fee5c01f020 con 0x7fee6415e9b0 2024-04-22T07:04:42.665 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:42.665 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1059009649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1059009649},{"type":"v1","addr":"172.21.15.145:6835","nonce":1059009649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:03.129095+0000","modified":"2024-04-22T07:04:31.038967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4275,"mds_1":4273,"mds_2":4269,"mds_3":4232,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.145:6836/1058270486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1058270486},{"type":"v1","addr":"172.21.15.145:6836","nonce":1058270486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4232":{"gid":4232,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.145:6837/3258302920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3258302920},{"type":"v1","addr":"172.21.15.145:6837","nonce":3258302920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/101798872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":101798872},{"type":"v1","addr":"172.21.15.92:6837","nonce":101798872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6839/3493022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3493022212},{"type":"v1","addr":"172.21.15.92:6839","nonce":3493022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4229,4232,4269,4273,4275]},"id":1}]} 2024-04-22T07:04:42.665 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2024-04-22T07:04:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.669+0000 7fee69274640 1 -- 172.21.15.92:0/2476389341 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fee5006c8f0 msgr2=0x7fee5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.669+0000 7fee69274640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fee5006c8f0 0x7fee5006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fee54035000 tx=0x7fee54030000 comp rx=0 tx=0).stop 2024-04-22T07:04:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.669+0000 7fee69274640 1 -- 172.21.15.92:0/2476389341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee6415e9b0 msgr2=0x7fee64156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.669+0000 7fee69274640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee6415e9b0 0x7fee64156440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00c7c0 tx=0x7fee5c00cc80 comp rx=0 tx=0).stop 2024-04-22T07:04:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.669+0000 7fee69274640 1 -- 172.21.15.92:0/2476389341 shutdown_connections 2024-04-22T07:04:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.669+0000 7fee69274640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fee5006c8f0 0x7fee5006edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.669+0000 7fee69274640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee6415e9b0 0x7fee64156440 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.669+0000 7fee69274640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee640d0f50 0x7fee640ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.669+0000 7fee69274640 1 --2- 172.21.15.92:0/2476389341 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee640d0b70 0x7fee640c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.669+0000 7fee69274640 1 -- 172.21.15.92:0/2476389341 >> 172.21.15.92:0/2476389341 conn(0x7fee640736b0 msgr2=0x7fee64159060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.669+0000 7fee69274640 1 -- 172.21.15.92:0/2476389341 shutdown_connections 2024-04-22T07:04:42.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.669+0000 7fee69274640 1 -- 172.21.15.92:0/2476389341 wait complete. 2024-04-22T07:04:42.678 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:42.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.777+0000 7f5909732640 1 Processor -- start 2024-04-22T07:04:42.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.777+0000 7f5909732640 1 -- start start 2024-04-22T07:04:42.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.777+0000 7f5909732640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f59040d0b70 0x7f5904154a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.777+0000 7f5909732640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f590415ab30 con 0x7f590415e9b0 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.777+0000 7f5909732640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f590415adf0 con 0x7f59040d0f50 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.777+0000 7f5909732640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f590415ba00 con 0x7f59040d0b70 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.777+0000 7f59037fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f590415e9b0 0x7f5904165430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48322/0 (socket says 172.21.15.92:48322) 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f59037fe640 1 -- 172.21.15.92:0/2493198432 learned_addr learned my addr 172.21.15.92:0/2493198432 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3066874041 0 0) 0x7f590415ab30 con 0x7f590415e9b0 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5902ffd640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f59040d0b70 0x7f5904154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58ec003520 con 0x7f590415e9b0 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2153654937 0 0) 0x7f58ec003520 con 0x7f590415e9b0 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f59040d0b70 msgr2=0x7f5904154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f59040d0b70 0x7f5904154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 >> v1:172.21.15.145:6789/0 conn(0x7f59040d0f50 legacy=0x7f590415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590415b0d0 con 0x7f590415e9b0 2024-04-22T07:04:42.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f58fc0019f0 con 0x7f590415e9b0 2024-04-22T07:04:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 >> v1:172.21.15.92:6789/0 conn(0x7f590415e9b0 legacy=0x7f5904165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58ec0039e0 0x7f58ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f59040d0b70 0x7f58ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58ec004d40 0x7f58ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f590415ba00 con 0x7f58ec0039e0 2024-04-22T07:04:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec00baf0 con 0x7f58ec004d40 2024-04-22T07:04:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f58ec00bcf0 con 0x7f59040d0b70 2024-04-22T07:04:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5902ffd640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58ec0039e0 0x7f58ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f59027fc640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f59040d0b70 0x7f58ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f59037fe640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58ec004d40 0x7f58ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5902ffd640 1 -- 172.21.15.92:0/2493198432 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f59040d0b70 msgr2=0x7f58ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5902ffd640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f59040d0b70 0x7f58ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5902ffd640 1 -- 172.21.15.92:0/2493198432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58ec004d40 msgr2=0x7f58ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5902ffd640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58ec004d40 0x7f58ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f5902ffd640 1 -- 172.21.15.92:0/2493198432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec003520 con 0x7f58ec0039e0 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.778+0000 7f59037fe640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58ec004d40 0x7f58ec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.779+0000 7f5902ffd640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58ec0039e0 0x7f58ec003e00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009ac0 tx=0x7f58ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d94ee724312eb30 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.779+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8039070 con 0x7f58ec0039e0 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.779+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f802c600 con 0x7f58ec0039e0 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.779+0000 7f5901ffb640 1 -- 172.21.15.92:0/2493198432 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8009690 con 0x7f58ec0039e0 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.780+0000 7f5909732640 1 -- 172.21.15.92:0/2493198432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58ec0039e0 msgr2=0x7f58ec003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.780+0000 7f5909732640 1 --2- 172.21.15.92:0/2493198432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58ec0039e0 0x7f58ec003e00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009ac0 tx=0x7f58ec005830 comp rx=0 tx=0).stop 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.780+0000 7f59037fe640 1 -- 172.21.15.92:0/2493198432 reap_dead start 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.780+0000 7f5909732640 1 -- 172.21.15.92:0/2493198432 shutdown_connections 2024-04-22T07:04:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.780+0000 7f5909732640 1 -- 172.21.15.92:0/2493198432 >> 172.21.15.92:0/2493198432 conn(0x7f59040736b0 msgr2=0x7f5904073ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.780+0000 7f5909732640 1 -- 172.21.15.92:0/2493198432 shutdown_connections 2024-04-22T07:04:42.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.780+0000 7f5909732640 1 -- 172.21.15.92:0/2493198432 wait complete. 2024-04-22T07:04:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.781+0000 7f5909732640 1 Processor -- start 2024-04-22T07:04:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.781+0000 7f5909732640 1 -- start start 2024-04-22T07:04:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f5909732640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f58ec0039e0 0x7f59041f4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f5909732640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59040ca7e0 0x7f59041490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f5909732640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f590414f510 0x7f590414f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f5909732640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f59040cae00 con 0x7f59040ca7e0 2024-04-22T07:04:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f5909732640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f590415b690 con 0x7f590414f510 2024-04-22T07:04:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f5909732640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f59040caf80 con 0x7f58ec0039e0 2024-04-22T07:04:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f59027fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59040ca7e0 0x7f59041490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f59027fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59040ca7e0 0x7f59041490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51578/0 (socket says 172.21.15.92:51578) 2024-04-22T07:04:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f59027fc640 1 -- 172.21.15.92:0/1995092053 learned_addr learned my addr 172.21.15.92:0/1995092053 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:42.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f5902ffd640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f58ec0039e0 0x7f59041f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f59037fe640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f590414f510 0x7f590414f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f59027fc640 1 -- 172.21.15.92:0/1995092053 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f58ec0039e0 msgr2=0x7f59041f4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f59027fc640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f58ec0039e0 0x7f59041f4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f59027fc640 1 -- 172.21.15.92:0/1995092053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f590414f510 msgr2=0x7f590414f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f59027fc640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f590414f510 0x7f590414f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f59027fc640 1 -- 172.21.15.92:0/1995092053 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59041f11f0 con 0x7f59040ca7e0 2024-04-22T07:04:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f59037fe640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f590414f510 0x7f590414f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.782+0000 7f59027fc640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59040ca7e0 0x7f59041490f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f58f400e550 tx=0x7f58f400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.783+0000 7f58eb7fe640 1 -- 172.21.15.92:0/1995092053 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f40052e0 con 0x7f59040ca7e0 2024-04-22T07:04:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.783+0000 7f58eb7fe640 1 -- 172.21.15.92:0/1995092053 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f4018070 con 0x7f59040ca7e0 2024-04-22T07:04:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.783+0000 7f5909732640 1 -- 172.21.15.92:0/1995092053 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59041f1490 con 0x7f59040ca7e0 2024-04-22T07:04:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.783+0000 7f58eb7fe640 1 -- 172.21.15.92:0/1995092053 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4003810 con 0x7f59040ca7e0 2024-04-22T07:04:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.783+0000 7f5909732640 1 -- 172.21.15.92:0/1995092053 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59041f3f30 con 0x7f59040ca7e0 2024-04-22T07:04:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.783+0000 7f5909732640 1 -- 172.21.15.92:0/1995092053 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5904069930 con 0x7f59040ca7e0 2024-04-22T07:04:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.784+0000 7f58eb7fe640 1 -- 172.21.15.92:0/1995092053 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58f40039b0 con 0x7f59040ca7e0 2024-04-22T07:04:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.784+0000 7f58eb7fe640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f58f006c920 0x7f58f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.784+0000 7f58eb7fe640 1 -- 172.21.15.92:0/1995092053 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f58f408b6e0 con 0x7f59040ca7e0 2024-04-22T07:04:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.784+0000 7f5902ffd640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f58f006c920 0x7f58f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.785+0000 7f5902ffd640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f58f006c920 0x7f58f006ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58f8036000 tx=0x7f58f8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.789+0000 7f58eb7fe640 1 -- 172.21.15.92:0/1995092053 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f58f4059ba0 con 0x7f59040ca7e0 2024-04-22T07:04:42.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.967+0000 7f5909732640 1 -- 172.21.15.92:0/1995092053 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59040b40d0 con 0x7f59040ca7e0 2024-04-22T07:04:42.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.968+0000 7f58eb7fe640 1 -- 172.21.15.92:0/1995092053 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f58f40594a0 con 0x7f59040ca7e0 2024-04-22T07:04:42.966 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:42.966 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:04:04.784969+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:42.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.970+0000 7f5909732640 1 -- 172.21.15.92:0/1995092053 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f58f006c920 msgr2=0x7f58f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.970+0000 7f5909732640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f58f006c920 0x7f58f006ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58f8036000 tx=0x7f58f8030000 comp rx=0 tx=0).stop 2024-04-22T07:04:42.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.970+0000 7f5909732640 1 -- 172.21.15.92:0/1995092053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59040ca7e0 msgr2=0x7f59041490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.970+0000 7f5909732640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59040ca7e0 0x7f59041490f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f58f400e550 tx=0x7f58f400ea10 comp rx=0 tx=0).stop 2024-04-22T07:04:42.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.970+0000 7f5909732640 1 -- 172.21.15.92:0/1995092053 shutdown_connections 2024-04-22T07:04:42.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.970+0000 7f5909732640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f58f006c920 0x7f58f006ede0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.970+0000 7f5909732640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f590414f510 0x7f590414f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.970+0000 7f5909732640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59040ca7e0 0x7f59041490f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.970+0000 7f5909732640 1 --2- 172.21.15.92:0/1995092053 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f58ec0039e0 0x7f59041f4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.970+0000 7f5909732640 1 -- 172.21.15.92:0/1995092053 >> 172.21.15.92:0/1995092053 conn(0x7f59040736b0 msgr2=0x7f59040cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.970+0000 7f5909732640 1 -- 172.21.15.92:0/1995092053 shutdown_connections 2024-04-22T07:04:42.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:42.970+0000 7f5909732640 1 -- 172.21.15.92:0/1995092053 wait complete. 2024-04-22T07:04:42.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:03:57.947601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:01.058111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:02.085602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:42.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:43.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.081+0000 7f8e24f67640 1 Processor -- start 2024-04-22T07:04:43.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.081+0000 7f8e24f67640 1 -- start start 2024-04-22T07:04:43.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.081+0000 7f8e24f67640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8e200c7ca0 0x7f8e2015b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.081+0000 7f8e24f67640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e2015af60 con 0x7f8e20071090 2024-04-22T07:04:43.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.081+0000 7f8e24f67640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e2006fc40 con 0x7f8e20070cb0 2024-04-22T07:04:43.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.081+0000 7f8e24f67640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f8e2006ff20 con 0x7f8e200c7ca0 2024-04-22T07:04:43.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.082+0000 7f8e1ed9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8e200c7ca0 0x7f8e2015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.082+0000 7f8e1ed9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8e200c7ca0 0x7f8e2015b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51588/0 (socket says 172.21.15.92:51588) 2024-04-22T07:04:43.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.082+0000 7f8e15d9b640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8e20071090 0x7f8e201517e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48332/0 (socket says 172.21.15.92:48332) 2024-04-22T07:04:43.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.082+0000 7f8e1ed9d640 1 -- 172.21.15.92:0/4062236458 learned_addr learned my addr 172.21.15.92:0/4062236458 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:43.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.082+0000 7f8e1ed9d640 1 -- 172.21.15.92:0/4062236458 >> v1:172.21.15.145:6789/0 conn(0x7f8e20070cb0 legacy=0x7f8e200c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.082+0000 7f8e1ed9d640 1 -- 172.21.15.92:0/4062236458 >> v1:172.21.15.92:6789/0 conn(0x7f8e20071090 legacy=0x7f8e201517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.082+0000 7f8e1ed9d640 1 -- 172.21.15.92:0/4062236458 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e20070200 con 0x7f8e200c7ca0 2024-04-22T07:04:43.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.083+0000 7f8e1ed9d640 1 --2- 172.21.15.92:0/4062236458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e200c7ca0 0x7f8e2015b440 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009e50 tx=0x7f8e1802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b003a5a23ed28890 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.084+0000 7f8e1dd9b640 1 -- 172.21.15.92:0/4062236458 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1802cbb0 con 0x7f8e200c7ca0 2024-04-22T07:04:43.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.084+0000 7f8e1dd9b640 1 -- 172.21.15.92:0/4062236458 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1802cd50 con 0x7f8e200c7ca0 2024-04-22T07:04:43.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.084+0000 7f8e1dd9b640 1 -- 172.21.15.92:0/4062236458 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18034610 con 0x7f8e200c7ca0 2024-04-22T07:04:43.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.085+0000 7f8e24f67640 1 -- 172.21.15.92:0/4062236458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e200c7ca0 msgr2=0x7f8e2015b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.085+0000 7f8e24f67640 1 --2- 172.21.15.92:0/4062236458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e200c7ca0 0x7f8e2015b440 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009e50 tx=0x7f8e1802c010 comp rx=0 tx=0).stop 2024-04-22T07:04:43.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.085+0000 7f8e24f67640 1 -- 172.21.15.92:0/4062236458 shutdown_connections 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.085+0000 7f8e24f67640 1 --2- 172.21.15.92:0/4062236458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e200c7ca0 0x7f8e2015b440 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.085+0000 7f8e24f67640 1 -- 172.21.15.92:0/4062236458 >> 172.21.15.92:0/4062236458 conn(0x7f8e200cd2a0 msgr2=0x7f8e200cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.085+0000 7f8e24f67640 1 -- 172.21.15.92:0/4062236458 shutdown_connections 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.085+0000 7f8e24f67640 1 -- 172.21.15.92:0/4062236458 wait complete. 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.086+0000 7f8e24f67640 1 Processor -- start 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.087+0000 7f8e24f67640 1 -- start start 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.087+0000 7f8e24f67640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e20070cb0 0x7f8e2014dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.087+0000 7f8e24f67640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e20071090 0x7f8e2014e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.087+0000 7f8e24f67640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e200c7ca0 0x7f8e20145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.087+0000 7f8e24f67640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2006d140 con 0x7f8e20070cb0 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.087+0000 7f8e24f67640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e200707c0 con 0x7f8e20071090 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.087+0000 7f8e24f67640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8e2006d2c0 con 0x7f8e200c7ca0 2024-04-22T07:04:43.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.087+0000 7f8e1e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e20070cb0 0x7f8e2014dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.087+0000 7f8e1e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e20070cb0 0x7f8e2014dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51594/0 (socket says 172.21.15.92:51594) 2024-04-22T07:04:43.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.087+0000 7f8e1e59c640 1 -- 172.21.15.92:0/1314768306 learned_addr learned my addr 172.21.15.92:0/1314768306 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:43.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.087+0000 7f8e15d9b640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e20071090 0x7f8e2014e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.087+0000 7f8e1ed9d640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e200c7ca0 0x7f8e20145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.088+0000 7f8e1e59c640 1 -- 172.21.15.92:0/1314768306 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e200c7ca0 msgr2=0x7f8e20145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.088+0000 7f8e1e59c640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e200c7ca0 0x7f8e20145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.088+0000 7f8e1e59c640 1 -- 172.21.15.92:0/1314768306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e20071090 msgr2=0x7f8e2014e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.088+0000 7f8e1e59c640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e20071090 0x7f8e2014e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.088+0000 7f8e1e59c640 1 -- 172.21.15.92:0/1314768306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e201eeec0 con 0x7f8e20070cb0 2024-04-22T07:04:43.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.088+0000 7f8e1e59c640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e20070cb0 0x7f8e2014dc90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000c430 tx=0x7f8e1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.088+0000 7f8e177fe640 1 -- 172.21.15.92:0/1314768306 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e100052d0 con 0x7f8e20070cb0 2024-04-22T07:04:43.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.088+0000 7f8e177fe640 1 -- 172.21.15.92:0/1314768306 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e10005490 con 0x7f8e20070cb0 2024-04-22T07:04:43.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.088+0000 7f8e177fe640 1 -- 172.21.15.92:0/1314768306 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10007260 con 0x7f8e20070cb0 2024-04-22T07:04:43.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.088+0000 7f8e24f67640 1 -- 172.21.15.92:0/1314768306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e201ef180 con 0x7f8e20070cb0 2024-04-22T07:04:43.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.088+0000 7f8e24f67640 1 -- 172.21.15.92:0/1314768306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2014c5a0 con 0x7f8e20070cb0 2024-04-22T07:04:43.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.089+0000 7f8e24f67640 1 -- 172.21.15.92:0/1314768306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e20068b00 con 0x7f8e20070cb0 2024-04-22T07:04:43.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.089+0000 7f8e177fe640 1 -- 172.21.15.92:0/1314768306 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e10007400 con 0x7f8e20070cb0 2024-04-22T07:04:43.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.090+0000 7f8e177fe640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8e0006c8f0 0x7f8e0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.090+0000 7f8e177fe640 1 -- 172.21.15.92:0/1314768306 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8e1008b000 con 0x7f8e20070cb0 2024-04-22T07:04:43.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.090+0000 7f8e15d9b640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8e0006c8f0 0x7f8e0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.090+0000 7f8e15d9b640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8e0006c8f0 0x7f8e0006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e2014fbc0 tx=0x7f8e08007e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.094+0000 7f8e177fe640 1 -- 172.21.15.92:0/1314768306 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e10058de0 con 0x7f8e20070cb0 2024-04-22T07:04:43.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.267+0000 7f8e24f67640 1 -- 172.21.15.92:0/1314768306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8e200c8380 con 0x7f8e20070cb0 2024-04-22T07:04:43.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.268+0000 7f8e177fe640 1 -- 172.21.15.92:0/1314768306 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f8e1005d0e0 con 0x7f8e20070cb0 2024-04-22T07:04:43.266 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-04-22T07:04:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.270+0000 7f8e24f67640 1 -- 172.21.15.92:0/1314768306 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8e0006c8f0 msgr2=0x7f8e0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.270+0000 7f8e24f67640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8e0006c8f0 0x7f8e0006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e2014fbc0 tx=0x7f8e08007e00 comp rx=0 tx=0).stop 2024-04-22T07:04:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.270+0000 7f8e24f67640 1 -- 172.21.15.92:0/1314768306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e20070cb0 msgr2=0x7f8e2014dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.270+0000 7f8e24f67640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e20070cb0 0x7f8e2014dc90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000c430 tx=0x7f8e1000c8f0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.270+0000 7f8e24f67640 1 -- 172.21.15.92:0/1314768306 shutdown_connections 2024-04-22T07:04:43.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.270+0000 7f8e24f67640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8e0006c8f0 0x7f8e0006edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.270+0000 7f8e24f67640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e200c7ca0 0x7f8e20145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.270+0000 7f8e24f67640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e20071090 0x7f8e2014e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.270+0000 7f8e24f67640 1 --2- 172.21.15.92:0/1314768306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e20070cb0 0x7f8e2014dc90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.270+0000 7f8e24f67640 1 -- 172.21.15.92:0/1314768306 >> 172.21.15.92:0/1314768306 conn(0x7f8e200cd2a0 msgr2=0x7f8e200c47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.270+0000 7f8e24f67640 1 -- 172.21.15.92:0/1314768306 shutdown_connections 2024-04-22T07:04:43.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.270+0000 7f8e24f67640 1 -- 172.21.15.92:0/1314768306 wait complete. 2024-04-22T07:04:43.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:04:43.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.379+0000 7fcf7f9db640 1 Processor -- start 2024-04-22T07:04:43.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.379+0000 7fcf7f9db640 1 -- start start 2024-04-22T07:04:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.380+0000 7fcf7f9db640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcf78154d40 0x7fcf7806c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.380+0000 7fcf7f9db640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf7815bb30 con 0x7fcf7806b970 2024-04-22T07:04:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.380+0000 7fcf7f9db640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf78068140 con 0x7fcf7806c7f0 2024-04-22T07:04:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.380+0000 7fcf7f9db640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcf7815b6a0 con 0x7fcf78154d40 2024-04-22T07:04:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.380+0000 7fcf7df78640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcf7806b970 0x7fcf780c8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48334/0 (socket says 172.21.15.92:48334) 2024-04-22T07:04:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.380+0000 7fcf7df78640 1 -- 172.21.15.92:0/3696947807 learned_addr learned my addr 172.21.15.92:0/3696947807 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.380+0000 7fcf7d777640 1 --2- 172.21.15.92:0/3696947807 >> v2:172.21.15.145:3300/0 conn(0x7fcf78154d40 0x7fcf7806c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.380+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619889337 0 0) 0x7fcf7815bb30 con 0x7fcf7806b970 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.380+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf64003520 con 0x7fcf7806b970 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3551918114 0 0) 0x7fcf64003520 con 0x7fcf7806b970 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 >> v2:172.21.15.145:3300/0 conn(0x7fcf78154d40 msgr2=0x7fcf7806c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 --2- 172.21.15.92:0/3696947807 >> v2:172.21.15.145:3300/0 conn(0x7fcf78154d40 0x7fcf7806c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 >> v1:172.21.15.145:6789/0 conn(0x7fcf7806c7f0 legacy=0x7fcf780676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7815ad70 con 0x7fcf7806b970 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7fcf70001670 con 0x7fcf7806b970 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 >> v1:172.21.15.92:6789/0 conn(0x7fcf7806b970 legacy=0x7fcf780c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 --2- 172.21.15.92:0/3696947807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 0x7fcf64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 --2- 172.21.15.92:0/3696947807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf64004800 0x7fcf64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 --2- 172.21.15.92:0/3696947807 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf640051c0 0x7fcf6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7815b6a0 con 0x7fcf640039b0 2024-04-22T07:04:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6400c070 con 0x7fcf64004800 2024-04-22T07:04:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6400c270 con 0x7fcf640051c0 2024-04-22T07:04:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf7d777640 1 --2- 172.21.15.92:0/3696947807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 0x7fcf64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf7cf76640 1 --2- 172.21.15.92:0/3696947807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf64004800 0x7fcf64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf7df78640 1 --2- 172.21.15.92:0/3696947807 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf640051c0 0x7fcf6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf7d777640 1 -- 172.21.15.92:0/3696947807 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf640051c0 msgr2=0x7fcf6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf7d777640 1 --2- 172.21.15.92:0/3696947807 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf640051c0 0x7fcf6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf7d777640 1 -- 172.21.15.92:0/3696947807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf64004800 msgr2=0x7fcf64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf7d777640 1 --2- 172.21.15.92:0/3696947807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf64004800 0x7fcf64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf7d777640 1 -- 172.21.15.92:0/3696947807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf64003520 con 0x7fcf640039b0 2024-04-22T07:04:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf7df78640 1 -- 172.21.15.92:0/3696947807 reap_dead start 2024-04-22T07:04:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.381+0000 7fcf7cf76640 1 --2- 172.21.15.92:0/3696947807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf64004800 0x7fcf64004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:43.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.382+0000 7fcf7d777640 1 --2- 172.21.15.92:0/3696947807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 0x7fcf64003dd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009fc0 tx=0x7fcf64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29783634807e7ed server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.382+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74039070 con 0x7fcf640039b0 2024-04-22T07:04:43.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.382+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7402cc20 con 0x7fcf640039b0 2024-04-22T07:04:43.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.382+0000 7fcf6ffff640 1 -- 172.21.15.92:0/3696947807 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74009c10 con 0x7fcf640039b0 2024-04-22T07:04:43.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.382+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3696947807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 msgr2=0x7fcf64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.382+0000 7fcf7f9db640 1 --2- 172.21.15.92:0/3696947807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 0x7fcf64003dd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009fc0 tx=0x7fcf64005830 comp rx=0 tx=0).stop 2024-04-22T07:04:43.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.383+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3696947807 shutdown_connections 2024-04-22T07:04:43.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.383+0000 7fcf7f9db640 1 --2- 172.21.15.92:0/3696947807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 0x7fcf64003dd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.383+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3696947807 >> 172.21.15.92:0/3696947807 conn(0x7fcf780cd440 msgr2=0x7fcf780cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.383+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3696947807 shutdown_connections 2024-04-22T07:04:43.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.383+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3696947807 wait complete. 2024-04-22T07:04:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.384+0000 7fcf7f9db640 1 Processor -- start 2024-04-22T07:04:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.384+0000 7fcf7f9db640 1 -- start start 2024-04-22T07:04:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.384+0000 7fcf7f9db640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 0x7fcf78151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.384+0000 7fcf7f9db640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf781f40f0 0x7fcf7814ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.384+0000 7fcf7f9db640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf7814db30 0x7fcf7814dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.384+0000 7fcf7f9db640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78155120 con 0x7fcf640039b0 2024-04-22T07:04:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.384+0000 7fcf7f9db640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7815b330 con 0x7fcf781f40f0 2024-04-22T07:04:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.384+0000 7fcf7f9db640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcf781552a0 con 0x7fcf7814db30 2024-04-22T07:04:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.384+0000 7fcf7d777640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 0x7fcf78151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf7d777640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 0x7fcf78151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51612/0 (socket says 172.21.15.92:51612) 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf7d777640 1 -- 172.21.15.92:0/3344102904 learned_addr learned my addr 172.21.15.92:0/3344102904 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf7df78640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf7814db30 0x7fcf7814dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf7cf76640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf781f40f0 0x7fcf7814ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf7d777640 1 -- 172.21.15.92:0/3344102904 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf7814db30 msgr2=0x7fcf7814dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf7d777640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf7814db30 0x7fcf7814dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf7d777640 1 -- 172.21.15.92:0/3344102904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf781f40f0 msgr2=0x7fcf7814ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf7d777640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf781f40f0 0x7fcf7814ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf7d777640 1 -- 172.21.15.92:0/3344102904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf78152410 con 0x7fcf640039b0 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf7d777640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 0x7fcf78151150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400e200 tx=0x7fcf74009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf6dffb640 1 -- 172.21.15.92:0/3344102904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74039050 con 0x7fcf640039b0 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf6dffb640 1 -- 172.21.15.92:0/3344102904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7403a040 con 0x7fcf640039b0 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf6dffb640 1 -- 172.21.15.92:0/3344102904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74009c10 con 0x7fcf640039b0 2024-04-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3344102904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf78151800 con 0x7fcf640039b0 2024-04-22T07:04:43.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.385+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3344102904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf78151dc0 con 0x7fcf640039b0 2024-04-22T07:04:43.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.386+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3344102904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf78069930 con 0x7fcf640039b0 2024-04-22T07:04:43.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.387+0000 7fcf6dffb640 1 -- 172.21.15.92:0/3344102904 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf74047020 con 0x7fcf640039b0 2024-04-22T07:04:43.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.387+0000 7fcf6dffb640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcf5c06c8f0 0x7fcf5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.387+0000 7fcf6dffb640 1 -- 172.21.15.92:0/3344102904 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf740add40 con 0x7fcf640039b0 2024-04-22T07:04:43.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.387+0000 7fcf7cf76640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcf5c06c8f0 0x7fcf5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.387+0000 7fcf7cf76640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcf5c06c8f0 0x7fcf5c06edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf68000c00 tx=0x7fcf68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.391+0000 7fcf6dffb640 1 -- 172.21.15.92:0/3344102904 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf7404e020 con 0x7fcf640039b0 2024-04-22T07:04:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.563+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3344102904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcf7814aef0 con 0x7fcf640039b0 2024-04-22T07:04:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.563+0000 7fcf6dffb640 1 -- 172.21.15.92:0/3344102904 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fcf7407c0e0 con 0x7fcf640039b0 2024-04-22T07:04:43.562 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-04-22T07:04:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.565+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3344102904 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcf5c06c8f0 msgr2=0x7fcf5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.565+0000 7fcf7f9db640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcf5c06c8f0 0x7fcf5c06edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf68000c00 tx=0x7fcf68009040 comp rx=0 tx=0).stop 2024-04-22T07:04:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.565+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3344102904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 msgr2=0x7fcf78151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.565+0000 7fcf7f9db640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 0x7fcf78151150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400e200 tx=0x7fcf74009750 comp rx=0 tx=0).stop 2024-04-22T07:04:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.565+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3344102904 shutdown_connections 2024-04-22T07:04:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.565+0000 7fcf7f9db640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fcf5c06c8f0 0x7fcf5c06edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.565+0000 7fcf7f9db640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf7814db30 0x7fcf7814dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.565+0000 7fcf7f9db640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf781f40f0 0x7fcf7814ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.565+0000 7fcf7f9db640 1 --2- 172.21.15.92:0/3344102904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf640039b0 0x7fcf78151150 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.565+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3344102904 >> 172.21.15.92:0/3344102904 conn(0x7fcf780cd440 msgr2=0x7fcf7806cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.565+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3344102904 shutdown_connections 2024-04-22T07:04:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.565+0000 7fcf7f9db640 1 -- 172.21.15.92:0/3344102904 wait complete. 2024-04-22T07:04:43.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:04:43.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.676+0000 7f787a92e640 1 Processor -- start 2024-04-22T07:04:43.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.676+0000 7f787a92e640 1 -- start start 2024-04-22T07:04:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f787a92e640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f787414f830 0x7f78740c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f787a92e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740cdfd0 0x7f78740ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f787a92e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78740c35c0 con 0x7f78740cedb0 2024-04-22T07:04:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f787a92e640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f78740cd270 con 0x7f787414f830 2024-04-22T07:04:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f787a92e640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f78740c38f0 con 0x7f78740cdfd0 2024-04-22T07:04:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7878ecb640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f78740cedb0 0x7f78741589b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48340/0 (socket says 172.21.15.92:48340) 2024-04-22T07:04:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7878ecb640 1 -- 172.21.15.92:0/4273541009 learned_addr learned my addr 172.21.15.92:0/4273541009 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7873fff640 1 --2- 172.21.15.92:0/4273541009 >> v2:172.21.15.145:3300/0 conn(0x7f787414f830 0x7f78740c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f78737fe640 1 --2- 172.21.15.92:0/4273541009 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740cdfd0 0x7f78740ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1387986023 0 0) 0x7f78740c35c0 con 0x7f78740cedb0 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f785c003520 con 0x7f78740cedb0 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2906251750 0 0) 0x7f785c003520 con 0x7f78740cedb0 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740cdfd0 msgr2=0x7f78740ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 --2- 172.21.15.92:0/4273541009 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740cdfd0 0x7f78740ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 >> v2:172.21.15.145:3300/0 conn(0x7f787414f830 msgr2=0x7f78740c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 --2- 172.21.15.92:0/4273541009 >> v2:172.21.15.145:3300/0 conn(0x7f787414f830 0x7f78740c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78740c5580 con 0x7f78740cedb0 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f786c0019f0 con 0x7f78740cedb0 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 >> v1:172.21.15.92:6789/0 conn(0x7f78740cedb0 legacy=0x7f78741589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 --2- 172.21.15.92:0/4273541009 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740cdfd0 0x7f785c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 --2- 172.21.15.92:0/4273541009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f785c003f30 0x7f785c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 --2- 172.21.15.92:0/4273541009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f785c004d40 0x7f785c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78740cd270 con 0x7f785c004d40 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f78740c38f0 con 0x7f785c003f30 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.677+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f785c00b9f0 con 0x7f78740cdfd0 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.678+0000 7f7878ecb640 1 --2- 172.21.15.92:0/4273541009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f785c004d40 0x7f785c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.678+0000 7f7873fff640 1 --2- 172.21.15.92:0/4273541009 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740cdfd0 0x7f785c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.678+0000 7f78737fe640 1 --2- 172.21.15.92:0/4273541009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f785c003f30 0x7f785c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.678+0000 7f7878ecb640 1 -- 172.21.15.92:0/4273541009 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740cdfd0 msgr2=0x7f785c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.678+0000 7f7878ecb640 1 --2- 172.21.15.92:0/4273541009 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740cdfd0 0x7f785c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.678+0000 7f7878ecb640 1 -- 172.21.15.92:0/4273541009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f785c003f30 msgr2=0x7f785c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.678+0000 7f7878ecb640 1 --2- 172.21.15.92:0/4273541009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f785c003f30 0x7f785c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.678+0000 7f7878ecb640 1 -- 172.21.15.92:0/4273541009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785c003520 con 0x7f785c004d40 2024-04-22T07:04:43.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.679+0000 7f7878ecb640 1 --2- 172.21.15.92:0/4273541009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f785c004d40 0x7f785c0051c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f786c00a880 tx=0x7f785c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fe7ef8fe8be45b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.679+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c00a4a0 con 0x7f785c004d40 2024-04-22T07:04:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.679+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f786c034650 con 0x7f785c004d40 2024-04-22T07:04:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.679+0000 7f7872ffd640 1 -- 172.21.15.92:0/4273541009 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c034a40 con 0x7f785c004d40 2024-04-22T07:04:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.679+0000 7f787a92e640 1 -- 172.21.15.92:0/4273541009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f785c004d40 msgr2=0x7f785c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.679+0000 7f787a92e640 1 --2- 172.21.15.92:0/4273541009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f785c004d40 0x7f785c0051c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f786c00a880 tx=0x7f785c005830 comp rx=0 tx=0).stop 2024-04-22T07:04:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.679+0000 7f7878ecb640 1 -- 172.21.15.92:0/4273541009 reap_dead start 2024-04-22T07:04:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.679+0000 7f787a92e640 1 -- 172.21.15.92:0/4273541009 shutdown_connections 2024-04-22T07:04:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.679+0000 7f787a92e640 1 -- 172.21.15.92:0/4273541009 >> 172.21.15.92:0/4273541009 conn(0x7f787406dd60 msgr2=0x7f787406e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.679+0000 7f787a92e640 1 -- 172.21.15.92:0/4273541009 shutdown_connections 2024-04-22T07:04:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.679+0000 7f787a92e640 1 -- 172.21.15.92:0/4273541009 wait complete. 2024-04-22T07:04:43.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.680+0000 7f787a92e640 1 Processor -- start 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.681+0000 7f787a92e640 1 -- start start 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.681+0000 7f787a92e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f787414e900 0x7f78741efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.681+0000 7f787a92e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740b1eb0 0x7f78741f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.681+0000 7f787a92e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740b3780 0x7f787414ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.681+0000 7f787a92e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78740cd7a0 con 0x7f78740b1eb0 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.681+0000 7f787a92e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f78740b1180 con 0x7f787414e900 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.681+0000 7f787a92e640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f78740c5140 con 0x7f78740b3780 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.681+0000 7f78737fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740b1eb0 0x7f78741f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.681+0000 7f78737fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740b1eb0 0x7f78741f2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51620/0 (socket says 172.21.15.92:51620) 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.681+0000 7f78737fe640 1 -- 172.21.15.92:0/2044122101 learned_addr learned my addr 172.21.15.92:0/2044122101 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.681+0000 7f7873fff640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f787414e900 0x7f78741efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.681+0000 7f7878ecb640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740b3780 0x7f787414ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.682+0000 7f78737fe640 1 -- 172.21.15.92:0/2044122101 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740b3780 msgr2=0x7f787414ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.682+0000 7f78737fe640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740b3780 0x7f787414ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.682+0000 7f78737fe640 1 -- 172.21.15.92:0/2044122101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f787414e900 msgr2=0x7f78741efbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.682+0000 7f78737fe640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f787414e900 0x7f78741efbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.682+0000 7f78737fe640 1 -- 172.21.15.92:0/2044122101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7874156070 con 0x7f78740b1eb0 2024-04-22T07:04:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.682+0000 7f78737fe640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740b1eb0 0x7f78741f2730 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7868011990 tx=0x7f7868011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.682+0000 7f7870ff9640 1 -- 172.21.15.92:0/2044122101 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786800fa30 con 0x7f78740b1eb0 2024-04-22T07:04:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.682+0000 7f7870ff9640 1 -- 172.21.15.92:0/2044122101 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f786800fbd0 con 0x7f78740b1eb0 2024-04-22T07:04:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.682+0000 7f787a92e640 1 -- 172.21.15.92:0/2044122101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7874155dd0 con 0x7f78740b1eb0 2024-04-22T07:04:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.682+0000 7f7870ff9640 1 -- 172.21.15.92:0/2044122101 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868014440 con 0x7f78740b1eb0 2024-04-22T07:04:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.682+0000 7f787a92e640 1 -- 172.21.15.92:0/2044122101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7874158560 con 0x7f78740b1eb0 2024-04-22T07:04:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.683+0000 7f787a92e640 1 -- 172.21.15.92:0/2044122101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78740c8f00 con 0x7f78740b1eb0 2024-04-22T07:04:43.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.683+0000 7f7870ff9640 1 -- 172.21.15.92:0/2044122101 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7868012040 con 0x7f78740b1eb0 2024-04-22T07:04:43.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.684+0000 7f7870ff9640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f786006c8f0 0x7f786006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.684+0000 7f7870ff9640 1 -- 172.21.15.92:0/2044122101 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f786808c0a0 con 0x7f78740b1eb0 2024-04-22T07:04:43.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.684+0000 7f7873fff640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f786006c8f0 0x7f786006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.684+0000 7f7873fff640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f786006c8f0 0x7f786006edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7864009c40 tx=0x7f7864009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.688+0000 7f7870ff9640 1 -- 172.21.15.92:0/2044122101 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f78740c8f00 con 0x7f78740b1eb0 2024-04-22T07:04:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.866+0000 7f787a92e640 1 -- 172.21.15.92:0/2044122101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7874069930 con 0x7f78740b1eb0 2024-04-22T07:04:43.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.866+0000 7f7870ff9640 1 -- 172.21.15.92:0/2044122101 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f7868019030 con 0x7f78740b1eb0 2024-04-22T07:04:43.865 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-04-22T07:04:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.868+0000 7f787a92e640 1 -- 172.21.15.92:0/2044122101 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f786006c8f0 msgr2=0x7f786006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.868+0000 7f787a92e640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f786006c8f0 0x7f786006edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7864009c40 tx=0x7f7864009190 comp rx=0 tx=0).stop 2024-04-22T07:04:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.868+0000 7f787a92e640 1 -- 172.21.15.92:0/2044122101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740b1eb0 msgr2=0x7f78741f2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.868+0000 7f787a92e640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740b1eb0 0x7f78741f2730 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7868011990 tx=0x7f7868011d90 comp rx=0 tx=0).stop 2024-04-22T07:04:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.868+0000 7f787a92e640 1 -- 172.21.15.92:0/2044122101 shutdown_connections 2024-04-22T07:04:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.868+0000 7f787a92e640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f786006c8f0 0x7f786006edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.869+0000 7f787a92e640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78740b3780 0x7f787414ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.869+0000 7f787a92e640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740b1eb0 0x7f78741f2730 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.869+0000 7f787a92e640 1 --2- 172.21.15.92:0/2044122101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f787414e900 0x7f78741efbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.869+0000 7f787a92e640 1 -- 172.21.15.92:0/2044122101 >> 172.21.15.92:0/2044122101 conn(0x7f787406dd60 msgr2=0x7f7874071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.869+0000 7f787a92e640 1 -- 172.21.15.92:0/2044122101 shutdown_connections 2024-04-22T07:04:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.869+0000 7f787a92e640 1 -- 172.21.15.92:0/2044122101 wait complete. 2024-04-22T07:04:43.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:43.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.976+0000 7f214dcce640 1 Processor -- start 2024-04-22T07:04:43.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.976+0000 7f214dcce640 1 -- start start 2024-04-22T07:04:43.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.977+0000 7f214dcce640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f21480c9db0 0x7f214806f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.977+0000 7f214dcce640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21480c8e00 con 0x7f214815b4a0 2024-04-22T07:04:43.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.977+0000 7f214dcce640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21480c90e0 con 0x7f21480ce2f0 2024-04-22T07:04:43.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.977+0000 7f214dcce640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f21480c9a10 con 0x7f21480c9db0 2024-04-22T07:04:43.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.977+0000 7f2146ffd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f214815b4a0 0x7f2148163590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48352/0 (socket says 172.21.15.92:48352) 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.977+0000 7f2146ffd640 1 -- 172.21.15.92:0/862830038 learned_addr learned my addr 172.21.15.92:0/862830038 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.977+0000 7f2147fff640 1 --2- 172.21.15.92:0/862830038 >> v2:172.21.15.145:3300/0 conn(0x7f21480c9db0 0x7f214806f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.977+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3759342042 0 0) 0x7f21480c8e00 con 0x7f214815b4a0 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.977+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2130003520 con 0x7f214815b4a0 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2628614415 0 0) 0x7f2130003520 con 0x7f214815b4a0 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 >> v2:172.21.15.145:3300/0 conn(0x7f21480c9db0 msgr2=0x7f214806f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 --2- 172.21.15.92:0/862830038 >> v2:172.21.15.145:3300/0 conn(0x7f21480c9db0 0x7f214806f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 >> v1:172.21.15.145:6789/0 conn(0x7f21480ce2f0 legacy=0x7f214815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21480c93c0 con 0x7f214815b4a0 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f213c001670 con 0x7f214815b4a0 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f2147fff640 1 --2- 172.21.15.92:0/862830038 >> v2:172.21.15.145:3300/0 conn(0x7f21480c9db0 0x7f214806f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 >> v1:172.21.15.92:6789/0 conn(0x7f214815b4a0 legacy=0x7f2148163590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 --2- 172.21.15.92:0/862830038 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21300039b0 0x7f2130003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 --2- 172.21.15.92:0/862830038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2130004800 0x7f2130004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 --2- 172.21.15.92:0/862830038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21300051c0 0x7f213000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f21480c9a10 con 0x7f2130004800 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f213000c070 con 0x7f21300051c0 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f213000c270 con 0x7f21300039b0 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f2146ffd640 1 --2- 172.21.15.92:0/862830038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2130004800 0x7f2130004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f21477fe640 1 --2- 172.21.15.92:0/862830038 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21300039b0 0x7f2130003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f2147fff640 1 --2- 172.21.15.92:0/862830038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21300051c0 0x7f213000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f2146ffd640 1 -- 172.21.15.92:0/862830038 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21300039b0 msgr2=0x7f2130003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f2146ffd640 1 --2- 172.21.15.92:0/862830038 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21300039b0 0x7f2130003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f2146ffd640 1 -- 172.21.15.92:0/862830038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21300051c0 msgr2=0x7f213000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f2146ffd640 1 --2- 172.21.15.92:0/862830038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21300051c0 0x7f213000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f2146ffd640 1 -- 172.21.15.92:0/862830038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2130003520 con 0x7f2130004800 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f2147fff640 1 --2- 172.21.15.92:0/862830038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21300051c0 0x7f213000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.978+0000 7f2147fff640 1 -- 172.21.15.92:0/862830038 reap_dead start 2024-04-22T07:04:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.979+0000 7f2146ffd640 1 --2- 172.21.15.92:0/862830038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2130004800 0x7f2130004c80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f213c00c240 tx=0x7f2130005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67a3f6fadcf1caf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.979+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c007cf0 con 0x7f2130004800 2024-04-22T07:04:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.979+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f213c030070 con 0x7f2130004800 2024-04-22T07:04:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.979+0000 7f21467fc640 1 -- 172.21.15.92:0/862830038 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c005d70 con 0x7f2130004800 2024-04-22T07:04:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.979+0000 7f214dcce640 1 -- 172.21.15.92:0/862830038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2130004800 msgr2=0x7f2130004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.979+0000 7f214dcce640 1 --2- 172.21.15.92:0/862830038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2130004800 0x7f2130004c80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f213c00c240 tx=0x7f2130005830 comp rx=0 tx=0).stop 2024-04-22T07:04:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.979+0000 7f214dcce640 1 -- 172.21.15.92:0/862830038 shutdown_connections 2024-04-22T07:04:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.979+0000 7f214dcce640 1 --2- 172.21.15.92:0/862830038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2130004800 0x7f2130004c80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.979+0000 7f214dcce640 1 -- 172.21.15.92:0/862830038 >> 172.21.15.92:0/862830038 conn(0x7f214806db80 msgr2=0x7f214806df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.980+0000 7f214dcce640 1 -- 172.21.15.92:0/862830038 shutdown_connections 2024-04-22T07:04:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.980+0000 7f214dcce640 1 -- 172.21.15.92:0/862830038 wait complete. 2024-04-22T07:04:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.981+0000 7f214dcce640 1 Processor -- start 2024-04-22T07:04:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.981+0000 7f214dcce640 1 -- start start 2024-04-22T07:04:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.981+0000 7f214dcce640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2130004800 0x7f2148157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.981+0000 7f214dcce640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21480c1be0 0x7f21480cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.981+0000 7f214dcce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2148157900 0x7f2148148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.981+0000 7f214dcce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2148070010 con 0x7f2148157900 2024-04-22T07:04:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.981+0000 7f214dcce640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f214806d910 con 0x7f2130004800 2024-04-22T07:04:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.981+0000 7f214dcce640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f21480cbad0 con 0x7f21480c1be0 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f2147fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2148157900 0x7f2148148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f2147fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2148157900 0x7f2148148670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51642/0 (socket says 172.21.15.92:51642) 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f2147fff640 1 -- 172.21.15.92:0/521313833 learned_addr learned my addr 172.21.15.92:0/521313833 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f2146ffd640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21480c1be0 0x7f21480cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f21477fe640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2130004800 0x7f2148157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f2147fff640 1 -- 172.21.15.92:0/521313833 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21480c1be0 msgr2=0x7f21480cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f2147fff640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21480c1be0 0x7f21480cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f2147fff640 1 -- 172.21.15.92:0/521313833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2130004800 msgr2=0x7f2148157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f2147fff640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2130004800 0x7f2148157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f2147fff640 1 -- 172.21.15.92:0/521313833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214814f490 con 0x7f2148157900 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f21477fe640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2130004800 0x7f2148157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f2146ffd640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21480c1be0 0x7f21480cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f2147fff640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2148157900 0x7f2148148670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f214806f790 tx=0x7f2140009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f212ffff640 1 -- 172.21.15.92:0/521313833 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140011d40 con 0x7f2148157900 2024-04-22T07:04:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f212ffff640 1 -- 172.21.15.92:0/521313833 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214001c070 con 0x7f2148157900 2024-04-22T07:04:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f214dcce640 1 -- 172.21.15.92:0/521313833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214814f700 con 0x7f2148157900 2024-04-22T07:04:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f212ffff640 1 -- 172.21.15.92:0/521313833 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21400166b0 con 0x7f2148157900 2024-04-22T07:04:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.982+0000 7f214dcce640 1 -- 172.21.15.92:0/521313833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21481565c0 con 0x7f2148157900 2024-04-22T07:04:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.983+0000 7f214dcce640 1 -- 172.21.15.92:0/521313833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2148069930 con 0x7f2148157900 2024-04-22T07:04:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.984+0000 7f212ffff640 1 -- 172.21.15.92:0/521313833 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2140016850 con 0x7f2148157900 2024-04-22T07:04:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.984+0000 7f212ffff640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f213406c920 0x7f213406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.984+0000 7f212ffff640 1 -- 172.21.15.92:0/521313833 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f214008c710 con 0x7f2148157900 2024-04-22T07:04:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.984+0000 7f21477fe640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f213406c920 0x7f213406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.984+0000 7f21477fe640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f213406c920 0x7f213406ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2138000c00 tx=0x7f2138009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:43.988+0000 7f212ffff640 1 -- 172.21.15.92:0/521313833 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f214005aab0 con 0x7f2148157900 2024-04-22T07:04:44.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.202+0000 7f214dcce640 1 -- 172.21.15.92:0/521313833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f214806d0a0 con 0x7f2148157900 2024-04-22T07:04:44.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.203+0000 7f212ffff640 1 -- 172.21.15.92:0/521313833 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6913 (secure 0 0 0) 0x7f214005a3b0 con 0x7f2148157900 2024-04-22T07:04:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:44.201 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1059009649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1059009649},{"type":"v1","addr":"172.21.15.145:6835","nonce":1059009649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:03.129095+0000","modified":"2024-04-22T07:04:31.038967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4275,"mds_1":4273,"mds_2":4269,"mds_3":4232,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.145:6836/1058270486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1058270486},{"type":"v1","addr":"172.21.15.145:6836","nonce":1058270486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4232":{"gid":4232,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.145:6837/3258302920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3258302920},{"type":"v1","addr":"172.21.15.145:6837","nonce":3258302920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/101798872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":101798872},{"type":"v1","addr":"172.21.15.92:6837","nonce":101798872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6839/3493022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3493022212},{"type":"v1","addr":"172.21.15.92:6839","nonce":3493022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4229,4232,4269,4273,4275]},"id":1}]} 2024-04-22T07:04:44.201 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2024-04-22T07:04:44.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.205+0000 7f214dcce640 1 -- 172.21.15.92:0/521313833 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f213406c920 msgr2=0x7f213406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.205+0000 7f214dcce640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f213406c920 0x7f213406ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2138000c00 tx=0x7f2138009040 comp rx=0 tx=0).stop 2024-04-22T07:04:44.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.205+0000 7f214dcce640 1 -- 172.21.15.92:0/521313833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2148157900 msgr2=0x7f2148148670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.205+0000 7f214dcce640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2148157900 0x7f2148148670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f214806f790 tx=0x7f2140009f90 comp rx=0 tx=0).stop 2024-04-22T07:04:44.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.205+0000 7f214dcce640 1 -- 172.21.15.92:0/521313833 shutdown_connections 2024-04-22T07:04:44.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.205+0000 7f214dcce640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f213406c920 0x7f213406ede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.205+0000 7f214dcce640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2148157900 0x7f2148148670 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.205+0000 7f214dcce640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21480c1be0 0x7f21480cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.205+0000 7f214dcce640 1 --2- 172.21.15.92:0/521313833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2130004800 0x7f2148157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.205+0000 7f214dcce640 1 -- 172.21.15.92:0/521313833 >> 172.21.15.92:0/521313833 conn(0x7f214806db80 msgr2=0x7f21480cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:44.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.205+0000 7f214dcce640 1 -- 172.21.15.92:0/521313833 shutdown_connections 2024-04-22T07:04:44.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.205+0000 7f214dcce640 1 -- 172.21.15.92:0/521313833 wait complete. 2024-04-22T07:04:44.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:44.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.312+0000 7f767cb9f640 1 Processor -- start 2024-04-22T07:04:44.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.312+0000 7f767cb9f640 1 -- start start 2024-04-22T07:04:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f767cb9f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76780cd200 0x7f767815b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f767cb9f640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f76780cd5e0 0x7f7678154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f767cb9f640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f767815e660 con 0x7f76780cb6b0 2024-04-22T07:04:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f767cb9f640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f767815f480 con 0x7f76780cd200 2024-04-22T07:04:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f767cb9f640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f767815e900 con 0x7f76780cd5e0 2024-04-22T07:04:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f767659c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76780cd200 0x7f767815b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f767659c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76780cd200 0x7f767815b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51644/0 (socket says 172.21.15.92:51644) 2024-04-22T07:04:44.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f767659c640 1 -- 172.21.15.92:0/759717941 learned_addr learned my addr 172.21.15.92:0/759717941 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f7675d9b640 1 --2- 172.21.15.92:0/759717941 >> v2:172.21.15.145:3300/0 conn(0x7f76780cd5e0 0x7f7678154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f767659c640 1 -- 172.21.15.92:0/759717941 >> v2:172.21.15.145:3300/0 conn(0x7f76780cd5e0 msgr2=0x7f7678154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f767659c640 1 --2- 172.21.15.92:0/759717941 >> v2:172.21.15.145:3300/0 conn(0x7f76780cd5e0 0x7f7678154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f767659c640 1 -- 172.21.15.92:0/759717941 >> v1:172.21.15.145:6789/0 conn(0x7f76780cb6b0 legacy=0x7f76780b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f767659c640 1 -- 172.21.15.92:0/759717941 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767815ebe0 con 0x7f76780cd200 2024-04-22T07:04:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.313+0000 7f7675d9b640 1 --2- 172.21.15.92:0/759717941 >> v2:172.21.15.145:3300/0 conn(0x7f76780cd5e0 0x7f7678154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.314+0000 7f767659c640 1 --2- 172.21.15.92:0/759717941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76780cd200 0x7f767815b9b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f76780741b0 tx=0x7f767002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a414280dc2083a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.314+0000 7f767559a640 1 -- 172.21.15.92:0/759717941 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767002caf0 con 0x7f76780cd200 2024-04-22T07:04:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.314+0000 7f767559a640 1 -- 172.21.15.92:0/759717941 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767002cc90 con 0x7f76780cd200 2024-04-22T07:04:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.314+0000 7f767559a640 1 -- 172.21.15.92:0/759717941 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670034610 con 0x7f76780cd200 2024-04-22T07:04:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.315+0000 7f767cb9f640 1 -- 172.21.15.92:0/759717941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76780cd200 msgr2=0x7f767815b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.315+0000 7f767cb9f640 1 --2- 172.21.15.92:0/759717941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76780cd200 0x7f767815b9b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f76780741b0 tx=0x7f767002bf50 comp rx=0 tx=0).stop 2024-04-22T07:04:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.315+0000 7f767cb9f640 1 -- 172.21.15.92:0/759717941 shutdown_connections 2024-04-22T07:04:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.315+0000 7f767cb9f640 1 --2- 172.21.15.92:0/759717941 >> v2:172.21.15.145:3300/0 conn(0x7f76780cd5e0 0x7f7678154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.315+0000 7f767cb9f640 1 --2- 172.21.15.92:0/759717941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76780cd200 0x7f767815b9b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.315+0000 7f767cb9f640 1 -- 172.21.15.92:0/759717941 >> 172.21.15.92:0/759717941 conn(0x7f7678073bc0 msgr2=0x7f76780728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.315+0000 7f767cb9f640 1 -- 172.21.15.92:0/759717941 shutdown_connections 2024-04-22T07:04:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.315+0000 7f767cb9f640 1 -- 172.21.15.92:0/759717941 wait complete. 2024-04-22T07:04:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.316+0000 7f767cb9f640 1 Processor -- start 2024-04-22T07:04:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.316+0000 7f767cb9f640 1 -- start start 2024-04-22T07:04:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f767cb9f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76780cb6b0 0x7f76780d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f767cb9f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76780cd200 0x7f76780c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f767cb9f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76780cd5e0 0x7f767815a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f767cb9f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f76780cc430 con 0x7f76780cd5e0 2024-04-22T07:04:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f767cb9f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f767815f1a0 con 0x7f76780cb6b0 2024-04-22T07:04:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f767cb9f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f76780cc5b0 con 0x7f76780cd200 2024-04-22T07:04:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7676d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76780cd5e0 0x7f767815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7676d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76780cd5e0 0x7f767815a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51652/0 (socket says 172.21.15.92:51652) 2024-04-22T07:04:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7676d9d640 1 -- 172.21.15.92:0/65710913 learned_addr learned my addr 172.21.15.92:0/65710913 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7675d9b640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76780cd200 0x7f76780c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f767659c640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76780cb6b0 0x7f76780d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7676d9d640 1 -- 172.21.15.92:0/65710913 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76780cd200 msgr2=0x7f76780c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7676d9d640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76780cd200 0x7f76780c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7676d9d640 1 -- 172.21.15.92:0/65710913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76780cb6b0 msgr2=0x7f76780d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7676d9d640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76780cb6b0 0x7f76780d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7676d9d640 1 -- 172.21.15.92:0/65710913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76780c96d0 con 0x7f76780cd5e0 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f767659c640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76780cb6b0 0x7f76780d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7676d9d640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76780cd5e0 0x7f767815a730 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f766800c440 tx=0x7f766800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7666ffd640 1 -- 172.21.15.92:0/65710913 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668011790 con 0x7f76780cd5e0 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7666ffd640 1 -- 172.21.15.92:0/65710913 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7668017070 con 0x7f76780cd5e0 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f767cb9f640 1 -- 172.21.15.92:0/65710913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76780c99b0 con 0x7f76780cd5e0 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.317+0000 7f7666ffd640 1 -- 172.21.15.92:0/65710913 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668011b70 con 0x7f76780cd5e0 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.318+0000 7f767cb9f640 1 -- 172.21.15.92:0/65710913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76780c9f70 con 0x7f76780cd5e0 2024-04-22T07:04:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.318+0000 7f767cb9f640 1 -- 172.21.15.92:0/65710913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76780c74b0 con 0x7f76780cd5e0 2024-04-22T07:04:44.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.319+0000 7f7666ffd640 1 -- 172.21.15.92:0/65710913 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7668011d10 con 0x7f76780cd5e0 2024-04-22T07:04:44.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.319+0000 7f7666ffd640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f765806c8f0 0x7f765806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.319+0000 7f7666ffd640 1 -- 172.21.15.92:0/65710913 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f766808b960 con 0x7f76780cd5e0 2024-04-22T07:04:44.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.319+0000 7f767659c640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f765806c8f0 0x7f765806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.320+0000 7f767659c640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f765806c8f0 0x7f765806edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7670035000 tx=0x7f7670030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:44.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.324+0000 7f7666ffd640 1 -- 172.21.15.92:0/65710913 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76780c74b0 con 0x7f76780cd5e0 2024-04-22T07:04:44.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.497+0000 7f767cb9f640 1 -- 172.21.15.92:0/65710913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f767806c800 con 0x7f76780cd5e0 2024-04-22T07:04:44.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.498+0000 7f7666ffd640 1 -- 172.21.15.92:0/65710913 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f767806c800 con 0x7f76780cd5e0 2024-04-22T07:04:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:44.497 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:04:04.784969+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:44.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.500+0000 7f767cb9f640 1 -- 172.21.15.92:0/65710913 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f765806c8f0 msgr2=0x7f765806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.500+0000 7f767cb9f640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f765806c8f0 0x7f765806edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7670035000 tx=0x7f7670030000 comp rx=0 tx=0).stop 2024-04-22T07:04:44.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.500+0000 7f767cb9f640 1 -- 172.21.15.92:0/65710913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76780cd5e0 msgr2=0x7f767815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.500+0000 7f767cb9f640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76780cd5e0 0x7f767815a730 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f766800c440 tx=0x7f766800c900 comp rx=0 tx=0).stop 2024-04-22T07:04:44.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.501+0000 7f767cb9f640 1 -- 172.21.15.92:0/65710913 shutdown_connections 2024-04-22T07:04:44.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.501+0000 7f767cb9f640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f765806c8f0 0x7f765806edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.501+0000 7f767cb9f640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76780cd5e0 0x7f767815a730 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.501+0000 7f767cb9f640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76780cd200 0x7f76780c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.501+0000 7f767cb9f640 1 --2- 172.21.15.92:0/65710913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76780cb6b0 0x7f76780d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.501+0000 7f767cb9f640 1 -- 172.21.15.92:0/65710913 >> 172.21.15.92:0/65710913 conn(0x7f7678073bc0 msgr2=0x7f767814ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:44.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.501+0000 7f767cb9f640 1 -- 172.21.15.92:0/65710913 shutdown_connections 2024-04-22T07:04:44.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.501+0000 7f767cb9f640 1 -- 172.21.15.92:0/65710913 wait complete. 2024-04-22T07:04:44.510 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 982869 2024-04-22T07:04:44.510 DEBUG:tasks.quiescer:Yielding 2024-04-22T07:04:44.510 INFO:teuthology.run_tasks:Running task workunit... 2024-04-22T07:04:44.524 INFO:tasks.workunit:Pulling workunits from ref 86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:04:44.524 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T07:04:44.524 INFO:tasks.workunit:timeout=6h 2024-04-22T07:04:44.524 INFO:tasks.workunit:cleanup=True 2024-04-22T07:04:44.525 DEBUG:teuthology.orchestra.run.smithi092:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:44.525 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-22T07:04:44.541 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-22T07:04:44.541 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:04:44.541 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-22T07:04:44.541 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-22T07:04:44.541 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-22 07:04:08.775617241 +0000 2024-04-22T07:04:44.541 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-22 07:04:38.874511854 +0000 2024-04-22T07:04:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-04-22T07:04:44.541 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:44.542 DEBUG:teuthology.orchestra.run.smithi092:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-22T07:04:44.611 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:04:44.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.627+0000 7fc7ef21f640 1 Processor -- start 2024-04-22T07:04:44.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.628+0000 7fc7ef21f640 1 -- start start 2024-04-22T07:04:44.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.628+0000 7fc7ef21f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7e8156190 0x7fc7e8070490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.628+0000 7fc7ef21f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e80720e0 con 0x7fc7e80c8df0 2024-04-22T07:04:44.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.628+0000 7fc7ef21f640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e8072350 con 0x7fc7e80c43c0 2024-04-22T07:04:44.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.628+0000 7fc7ef21f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e80c3410 con 0x7fc7e8156190 2024-04-22T07:04:44.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.628+0000 7fc7e7fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc7e80c8df0 0x7fc7e80c3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48368/0 (socket says 172.21.15.92:48368) 2024-04-22T07:04:44.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.628+0000 7fc7e7fff640 1 -- 172.21.15.92:0/1945128172 learned_addr learned my addr 172.21.15.92:0/1945128172 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:44.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164717610 0 0) 0x7fc7e80720e0 con 0x7fc7e80c8df0 2024-04-22T07:04:44.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7ecfbb640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7e8156190 0x7fc7e8070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7d8003520 con 0x7fc7e80c8df0 2024-04-22T07:04:44.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 480819915 0 0) 0x7fc7d8003520 con 0x7fc7e80c8df0 2024-04-22T07:04:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7e8156190 msgr2=0x7fc7e8070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7e8156190 0x7fc7e8070490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 >> v1:172.21.15.145:6789/0 conn(0x7fc7e80c43c0 legacy=0x7fc7e8163330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e80c3660 con 0x7fc7e80c8df0 2024-04-22T07:04:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7fc7d4001690 con 0x7fc7e80c8df0 2024-04-22T07:04:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 >> v1:172.21.15.92:6789/0 conn(0x7fc7e80c8df0 legacy=0x7fc7e80c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7d80039e0 0x7fc7d8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7d8004800 0x7fc7d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7e8156190 0x7fc7d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e80c3410 con 0x7fc7d80039e0 2024-04-22T07:04:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d800ba70 con 0x7fc7d8004800 2024-04-22T07:04:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc7d800bc70 con 0x7fc7e8156190 2024-04-22T07:04:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7ecfbb640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7d80039e0 0x7fc7d8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e7fff640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7d8004800 0x7fc7d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7ed7bc640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7e8156190 0x7fc7d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7ecfbb640 1 -- 172.21.15.92:0/1945128172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7e8156190 msgr2=0x7fc7d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:44.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7ecfbb640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7e8156190 0x7fc7d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7ecfbb640 1 -- 172.21.15.92:0/1945128172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7d8004800 msgr2=0x7fc7d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:44.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7ecfbb640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7d8004800 0x7fc7d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7ecfbb640 1 -- 172.21.15.92:0/1945128172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d8003520 con 0x7fc7d80039e0 2024-04-22T07:04:44.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.629+0000 7fc7e7fff640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7d8004800 0x7fc7d8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:44.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.630+0000 7fc7ecfbb640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7d80039e0 0x7fc7d8003e00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009a90 tx=0x7fc7d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceba404a90c01993 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:44.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.630+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc039070 con 0x7fc7d80039e0 2024-04-22T07:04:44.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.630+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7dc02c680 con 0x7fc7d80039e0 2024-04-22T07:04:44.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.630+0000 7fc7e77fe640 1 -- 172.21.15.92:0/1945128172 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc009690 con 0x7fc7d80039e0 2024-04-22T07:04:44.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.631+0000 7fc7ef21f640 1 -- 172.21.15.92:0/1945128172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7d80039e0 msgr2=0x7fc7d8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.631+0000 7fc7ef21f640 1 --2- 172.21.15.92:0/1945128172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7d80039e0 0x7fc7d8003e00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009a90 tx=0x7fc7d8005830 comp rx=0 tx=0).stop 2024-04-22T07:04:44.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.631+0000 7fc7ed7bc640 1 -- 172.21.15.92:0/1945128172 reap_dead start 2024-04-22T07:04:44.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.631+0000 7fc7ef21f640 1 -- 172.21.15.92:0/1945128172 shutdown_connections 2024-04-22T07:04:44.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.631+0000 7fc7ef21f640 1 -- 172.21.15.92:0/1945128172 >> 172.21.15.92:0/1945128172 conn(0x7fc7e806e260 msgr2=0x7fc7e80715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.631+0000 7fc7ef21f640 1 -- 172.21.15.92:0/1945128172 shutdown_connections 2024-04-22T07:04:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.631+0000 7fc7ef21f640 1 -- 172.21.15.92:0/1945128172 wait complete. 2024-04-22T07:04:44.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.632+0000 7fc7ef21f640 1 Processor -- start 2024-04-22T07:04:44.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.632+0000 7fc7ef21f640 1 -- start start 2024-04-22T07:04:44.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.633+0000 7fc7ef21f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7d80039e0 0x7fc7e814fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.633+0000 7fc7ef21f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7e80c84e0 0x7fc7e80cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.633+0000 7fc7ef21f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7e8151fc0 0x7fc7e81f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.633+0000 7fc7ef21f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e806f9b0 con 0x7fc7e80c84e0 2024-04-22T07:04:44.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.633+0000 7fc7ef21f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8070000 con 0x7fc7e8151fc0 2024-04-22T07:04:44.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.633+0000 7fc7ef21f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e806fc90 con 0x7fc7d80039e0 2024-04-22T07:04:44.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.633+0000 7fc7e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7e80c84e0 0x7fc7e80cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.633+0000 7fc7e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7e80c84e0 0x7fc7e80cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51664/0 (socket says 172.21.15.92:51664) 2024-04-22T07:04:44.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.633+0000 7fc7e7fff640 1 -- 172.21.15.92:0/3920142837 learned_addr learned my addr 172.21.15.92:0/3920142837 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:44.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7ecfbb640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7d80039e0 0x7fc7e814fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7ed7bc640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7e8151fc0 0x7fc7e81f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7e7fff640 1 -- 172.21.15.92:0/3920142837 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7d80039e0 msgr2=0x7fc7e814fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7e7fff640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7d80039e0 0x7fc7e814fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7e7fff640 1 -- 172.21.15.92:0/3920142837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7e8151fc0 msgr2=0x7fc7e81f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7e7fff640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7e8151fc0 0x7fc7e81f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7e7fff640 1 -- 172.21.15.92:0/3920142837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e80ce320 con 0x7fc7e80c84e0 2024-04-22T07:04:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7e7fff640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7e80c84e0 0x7fc7e80cd4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400e8a0 tx=0x7fc7d400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7e57fa640 1 -- 172.21.15.92:0/3920142837 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d400c870 con 0x7fc7e80c84e0 2024-04-22T07:04:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7e57fa640 1 -- 172.21.15.92:0/3920142837 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d4018070 con 0x7fc7e80c84e0 2024-04-22T07:04:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7ef21f640 1 -- 172.21.15.92:0/3920142837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e80ce600 con 0x7fc7e80c84e0 2024-04-22T07:04:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7e57fa640 1 -- 172.21.15.92:0/3920142837 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d400cc10 con 0x7fc7e80c84e0 2024-04-22T07:04:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.634+0000 7fc7ef21f640 1 -- 172.21.15.92:0/3920142837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e80cebc0 con 0x7fc7e80c84e0 2024-04-22T07:04:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.635+0000 7fc7ef21f640 1 -- 172.21.15.92:0/3920142837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e80b43e0 con 0x7fc7e80c84e0 2024-04-22T07:04:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.636+0000 7fc7e57fa640 1 -- 172.21.15.92:0/3920142837 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7d401d460 con 0x7fc7e80c84e0 2024-04-22T07:04:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.636+0000 7fc7e57fa640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc7cc06c8f0 0x7fc7cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.636+0000 7fc7ecfbb640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc7cc06c8f0 0x7fc7cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.636+0000 7fc7e57fa640 1 -- 172.21.15.92:0/3920142837 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc7d408b9f0 con 0x7fc7e80c84e0 2024-04-22T07:04:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.637+0000 7fc7ecfbb640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc7cc06c8f0 0x7fc7cc06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc036000 tx=0x7fc7dc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:44.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.640+0000 7fc7e57fa640 1 -- 172.21.15.92:0/3920142837 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7d4059e20 con 0x7fc7e80c84e0 2024-04-22T07:04:44.646 INFO:tasks.workunit.client.0.smithi092.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T07:04:44.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.865+0000 7fc7ef21f640 1 -- 172.21.15.92:0/3920142837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7e80c94b0 con 0x7fc7e80c84e0 2024-04-22T07:04:44.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.865+0000 7fc7e57fa640 1 -- 172.21.15.92:0/3920142837 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6913 (secure 0 0 0) 0x7fc7d4059720 con 0x7fc7e80c84e0 2024-04-22T07:04:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:04:44.864 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1059009649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1059009649},{"type":"v1","addr":"172.21.15.145:6835","nonce":1059009649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:03.129095+0000","modified":"2024-04-22T07:04:31.038967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4275,"mds_1":4273,"mds_2":4269,"mds_3":4232,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.145:6836/1058270486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1058270486},{"type":"v1","addr":"172.21.15.145:6836","nonce":1058270486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4232":{"gid":4232,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.145:6837/3258302920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3258302920},{"type":"v1","addr":"172.21.15.145:6837","nonce":3258302920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/101798872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":101798872},{"type":"v1","addr":"172.21.15.92:6837","nonce":101798872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6839/3493022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3493022212},{"type":"v1","addr":"172.21.15.92:6839","nonce":3493022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4229,4232,4269,4273,4275]},"id":1}]} 2024-04-22T07:04:44.864 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2024-04-22T07:04:44.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.867+0000 7fc7ef21f640 1 -- 172.21.15.92:0/3920142837 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc7cc06c8f0 msgr2=0x7fc7cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.867+0000 7fc7ef21f640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc7cc06c8f0 0x7fc7cc06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc036000 tx=0x7fc7dc030000 comp rx=0 tx=0).stop 2024-04-22T07:04:44.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.867+0000 7fc7ef21f640 1 -- 172.21.15.92:0/3920142837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7e80c84e0 msgr2=0x7fc7e80cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.867+0000 7fc7ef21f640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7e80c84e0 0x7fc7e80cd4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400e8a0 tx=0x7fc7d400ed60 comp rx=0 tx=0).stop 2024-04-22T07:04:44.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.867+0000 7fc7ef21f640 1 -- 172.21.15.92:0/3920142837 shutdown_connections 2024-04-22T07:04:44.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.867+0000 7fc7ef21f640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc7cc06c8f0 0x7fc7cc06edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.867+0000 7fc7ef21f640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7e8151fc0 0x7fc7e81f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.867+0000 7fc7ef21f640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7e80c84e0 0x7fc7e80cd4b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.867+0000 7fc7ef21f640 1 --2- 172.21.15.92:0/3920142837 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc7d80039e0 0x7fc7e814fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.867+0000 7fc7ef21f640 1 -- 172.21.15.92:0/3920142837 >> 172.21.15.92:0/3920142837 conn(0x7fc7e806e260 msgr2=0x7fc7e8155080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:44.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.867+0000 7fc7ef21f640 1 -- 172.21.15.92:0/3920142837 shutdown_connections 2024-04-22T07:04:44.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:44.867+0000 7fc7ef21f640 1 -- 172.21.15.92:0/3920142837 wait complete. 2024-04-22T07:04:45.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-22T07:04:03.129095+0000', 'modified': '2024-04-22T07:04:31.038967+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4275, 'mds_1': 4273, 'mds_2': 4269, 'mds_3': 4232, 'mds_4': 4229}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4229': {'gid': 4229, 'name': 'b', 'rank': 4, 'incarnation': 16, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.145:6836/1058270486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6833', 'nonce': 1058270486}, {'type': 'v1', 'addr': '172.21.15.145:6836', 'nonce': 1058270486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4232': {'gid': 4232, 'name': 'f', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.145:6837/3258302920', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 3258302920}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 3258302920}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4269': {'gid': 4269, 'name': 'c', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.92:6836/1444537065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 1444537065}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 1444537065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4273': {'gid': 4273, 'name': 'a', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6837/101798872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 101798872}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 101798872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4275': {'gid': 4275, 'name': 'e', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.92:6839/3493022212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 3493022212}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 3493022212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4275, 'qdb_cluster': [4229, 4232, 4269, 4273, 4275]} 2024-04-22T07:04:45.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-22T07:04:45.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T07:04:45.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.584+0000 7fa373375640 1 Processor -- start 2024-04-22T07:04:45.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.584+0000 7fa373375640 1 -- start start 2024-04-22T07:04:45.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa373375640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa36c151b70 0x7fa36c0c5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa373375640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa36c158090 0x7fa36c0c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa373375640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa36c0c87a0 con 0x7fa36c0c7840 2024-04-22T07:04:45.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa373375640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fa36c0c3720 con 0x7fa36c158090 2024-04-22T07:04:45.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa373375640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa36c0c3a00 con 0x7fa36c151b70 2024-04-22T07:04:45.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa371912640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa36c158090 0x7fa36c0c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa371912640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa36c158090 0x7fa36c0c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51670/0 (socket says 172.21.15.92:51670) 2024-04-22T07:04:45.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa371912640 1 -- 172.21.15.92:0/1311751196 learned_addr learned my addr 172.21.15.92:0/1311751196 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa371912640 1 -- 172.21.15.92:0/1311751196 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa36c151b70 msgr2=0x7fa36c0c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa371111640 1 --2- 172.21.15.92:0/1311751196 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa36c151b70 0x7fa36c0c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa371912640 1 --2- 172.21.15.92:0/1311751196 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa36c151b70 0x7fa36c0c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa371912640 1 -- 172.21.15.92:0/1311751196 >> v1:172.21.15.145:6789/0 conn(0x7fa36c0c7840 legacy=0x7fa36c157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.585+0000 7fa371912640 1 -- 172.21.15.92:0/1311751196 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36c06cdf0 con 0x7fa36c158090 2024-04-22T07:04:45.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.586+0000 7fa371912640 1 --2- 172.21.15.92:0/1311751196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa36c158090 0x7fa36c0c58a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa364009e50 tx=0x7fa36402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290ac194de2be178 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.586+0000 7fa363fff640 1 -- 172.21.15.92:0/1311751196 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36402cb00 con 0x7fa36c158090 2024-04-22T07:04:45.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.586+0000 7fa363fff640 1 -- 172.21.15.92:0/1311751196 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa36402cca0 con 0x7fa36c158090 2024-04-22T07:04:45.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.586+0000 7fa363fff640 1 -- 172.21.15.92:0/1311751196 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364034610 con 0x7fa36c158090 2024-04-22T07:04:45.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.587+0000 7fa373375640 1 -- 172.21.15.92:0/1311751196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa36c158090 msgr2=0x7fa36c0c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.587+0000 7fa373375640 1 --2- 172.21.15.92:0/1311751196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa36c158090 0x7fa36c0c58a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa364009e50 tx=0x7fa36402bf60 comp rx=0 tx=0).stop 2024-04-22T07:04:45.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.587+0000 7fa373375640 1 -- 172.21.15.92:0/1311751196 shutdown_connections 2024-04-22T07:04:45.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.587+0000 7fa373375640 1 --2- 172.21.15.92:0/1311751196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa36c158090 0x7fa36c0c58a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.587+0000 7fa373375640 1 --2- 172.21.15.92:0/1311751196 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa36c151b70 0x7fa36c0c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.587+0000 7fa373375640 1 -- 172.21.15.92:0/1311751196 >> 172.21.15.92:0/1311751196 conn(0x7fa36c0cc220 msgr2=0x7fa36c0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.587+0000 7fa373375640 1 -- 172.21.15.92:0/1311751196 shutdown_connections 2024-04-22T07:04:45.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.587+0000 7fa373375640 1 -- 172.21.15.92:0/1311751196 wait complete. 2024-04-22T07:04:45.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.588+0000 7fa373375640 1 Processor -- start 2024-04-22T07:04:45.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.588+0000 7fa373375640 1 -- start start 2024-04-22T07:04:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.588+0000 7fa373375640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa36c0c7840 0x7fa36c154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.588+0000 7fa373375640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa36c151b70 0x7fa36c1ef230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.588+0000 7fa373375640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa36c158090 0x7fa36c158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.588+0000 7fa373375640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c0c7190 con 0x7fa36c158090 2024-04-22T07:04:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.588+0000 7fa373375640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c0c74c0 con 0x7fa36c0c7840 2024-04-22T07:04:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.588+0000 7fa373375640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa36c151f50 con 0x7fa36c151b70 2024-04-22T07:04:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa371912640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa36c158090 0x7fa36c158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa371912640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa36c158090 0x7fa36c158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51684/0 (socket says 172.21.15.92:51684) 2024-04-22T07:04:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa371912640 1 -- 172.21.15.92:0/1630413288 learned_addr learned my addr 172.21.15.92:0/1630413288 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa370910640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa36c151b70 0x7fa36c1ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa371111640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa36c0c7840 0x7fa36c154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa371912640 1 -- 172.21.15.92:0/1630413288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa36c151b70 msgr2=0x7fa36c1ef230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa371912640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa36c151b70 0x7fa36c1ef230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa371912640 1 -- 172.21.15.92:0/1630413288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa36c0c7840 msgr2=0x7fa36c154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa371912640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa36c0c7840 0x7fa36c154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa371912640 1 -- 172.21.15.92:0/1630413288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36c1522f0 con 0x7fa36c158090 2024-04-22T07:04:45.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa371912640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa36c158090 0x7fa36c158d50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa364036040 tx=0x7fa364004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa361ffb640 1 -- 172.21.15.92:0/1630413288 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364044020 con 0x7fa36c158090 2024-04-22T07:04:45.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa361ffb640 1 -- 172.21.15.92:0/1630413288 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa36402ce50 con 0x7fa36c158090 2024-04-22T07:04:45.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa373375640 1 -- 172.21.15.92:0/1630413288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36c155090 con 0x7fa36c158090 2024-04-22T07:04:45.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa361ffb640 1 -- 172.21.15.92:0/1630413288 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364009510 con 0x7fa36c158090 2024-04-22T07:04:45.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.589+0000 7fa373375640 1 -- 172.21.15.92:0/1630413288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36c152870 con 0x7fa36c158090 2024-04-22T07:04:45.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.591+0000 7fa361ffb640 1 -- 172.21.15.92:0/1630413288 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa364047020 con 0x7fa36c158090 2024-04-22T07:04:45.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.591+0000 7fa361ffb640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa35006c8f0 0x7fa35006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.591+0000 7fa361ffb640 1 -- 172.21.15.92:0/1630413288 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3640ae5e0 con 0x7fa36c158090 2024-04-22T07:04:45.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.591+0000 7fa371111640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa35006c8f0 0x7fa35006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.592+0000 7fa371111640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa35006c8f0 0x7fa35006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa368006fd0 tx=0x7fa368008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.592+0000 7fa375bfd740 1 Processor -- start 2024-04-22T07:04:45.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.592+0000 7fa375bfd740 1 -- start start 2024-04-22T07:04:45.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa375bfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b35dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa375bfd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b35ddec0 0x55b9b3601d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa375bfd740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361d220 0x55b9b3602290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa375bfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b9b3540b10 con 0x55b9b361ce40 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa375bfd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b9b34de240 con 0x55b9b35ddec0 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa375bfd740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55b9b353cad0 con 0x55b9b361d220 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa371111640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b35dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa371111640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b35dc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51686/0 (socket says 172.21.15.92:51686) 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa371111640 1 -- 172.21.15.92:0/2121640134 learned_addr learned my addr 172.21.15.92:0/2121640134 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa371111640 1 -- 172.21.15.92:0/2121640134 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361d220 msgr2=0x55b9b3602290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa371111640 1 --2- 172.21.15.92:0/2121640134 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361d220 0x55b9b3602290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa371111640 1 -- 172.21.15.92:0/2121640134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b35ddec0 msgr2=0x55b9b3601d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa371111640 1 --2- 172.21.15.92:0/2121640134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b35ddec0 0x55b9b3601d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa371111640 1 -- 172.21.15.92:0/2121640134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b32532d0 con 0x55b9b361ce40 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa371111640 1 --2- 172.21.15.92:0/2121640134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b35dc1b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa36800db90 tx=0x7fa368010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c550124c028588da server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.593+0000 7fa356ffd640 1 -- 172.21.15.92:0/2121640134 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368019050 con 0x55b9b361ce40 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.594+0000 7fa356ffd640 1 -- 172.21.15.92:0/2121640134 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa368023070 con 0x55b9b361ce40 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.594+0000 7fa356ffd640 1 -- 172.21.15.92:0/2121640134 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368012d60 con 0x55b9b361ce40 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.594+0000 7fa375bfd740 1 -- 172.21.15.92:0/2121640134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 msgr2=0x55b9b35dc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.594+0000 7fa375bfd740 1 --2- 172.21.15.92:0/2121640134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b35dc1b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa36800db90 tx=0x7fa368010000 comp rx=0 tx=0).stop 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.594+0000 7fa375bfd740 1 -- 172.21.15.92:0/2121640134 shutdown_connections 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.594+0000 7fa375bfd740 1 --2- 172.21.15.92:0/2121640134 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361d220 0x55b9b3602290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.594+0000 7fa375bfd740 1 --2- 172.21.15.92:0/2121640134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b35ddec0 0x55b9b3601d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.594+0000 7fa375bfd740 1 --2- 172.21.15.92:0/2121640134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b35dc1b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.594+0000 7fa375bfd740 1 -- 172.21.15.92:0/2121640134 >> 172.21.15.92:0/2121640134 conn(0x55b9b35d8990 msgr2=0x55b9b35db290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.594+0000 7fa375bfd740 1 -- 172.21.15.92:0/2121640134 shutdown_connections 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.594+0000 7fa375bfd740 1 -- 172.21.15.92:0/2121640134 wait complete. 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.595+0000 7fa375bfd740 1 -- start start 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.595+0000 7fa375bfd740 1 Processor -- start 2024-04-22T07:04:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa375bfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b3605910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa375bfd740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361d220 0x55b9b36222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa375bfd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b36019e0 0x55b9b36234f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa375bfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b9b35d7220 con 0x55b9b361ce40 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa375bfd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b9b34c9190 con 0x55b9b36019e0 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa375bfd740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55b9b35d6540 con 0x55b9b361d220 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa375bfd740 10 client.0 _ll_register_callbacks cb 0x55b9b34c48b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa371111640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b3605910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa371111640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b3605910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51698/0 (socket says 172.21.15.92:51698) 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa371111640 1 -- 172.21.15.92:0/3267982273 learned_addr learned my addr 172.21.15.92:0/3267982273 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa370910640 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361d220 0x55b9b36222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa371111640 1 -- 172.21.15.92:0/3267982273 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361d220 msgr2=0x55b9b36222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa371111640 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361d220 0x55b9b36222b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa371111640 1 -- 172.21.15.92:0/3267982273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b36019e0 msgr2=0x55b9b36234f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa371912640 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b36019e0 0x55b9b36234f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa371111640 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b36019e0 0x55b9b36234f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa371111640 1 -- 172.21.15.92:0/3267982273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b331d6b0 con 0x55b9b361ce40 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa371111640 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b3605910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa368008010 tx=0x7fa368004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b415e473425efe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa34bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa34bfff640 1 -- 172.21.15.92:0/3267982273 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36801e990 con 0x55b9b361ce40 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa34bfff640 1 -- 172.21.15.92:0/3267982273 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa36801eb30 con 0x55b9b361ce40 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa375bfd740 1 -- 172.21.15.92:0/3267982273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa368023070 con 0x55b9b361ce40 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa34bfff640 1 -- 172.21.15.92:0/3267982273 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa36803b050 con 0x55b9b361ce40 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa375bfd740 10 client.4965 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.596+0000 7fa375bfd740 1 -- 172.21.15.92:0/3267982273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b9b3600f00 con 0x55b9b361ce40 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.597+0000 7fa34bfff640 1 -- 172.21.15.92:0/3267982273 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa368023070 con 0x55b9b361ce40 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.597+0000 7fa34bfff640 1 -- 172.21.15.92:0/3267982273 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fa36803e020 con 0x55b9b361ce40 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.597+0000 7fa375bfd740 10 client.4965 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.597+0000 7fa375bfd740 10 client.4965 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:04:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.597+0000 7fa375bfd740 20 client.4965 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:45.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.597+0000 7fa375bfd740 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55b9b363e960 0x55b9b3640d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.597+0000 7fa375bfd740 4 client.4965 mds_command: new command op to 4229 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:45.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.597+0000 7fa375bfd740 1 -- 172.21.15.92:0/3267982273 --> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9b354e530 con 0x55b9b363e960 2024-04-22T07:04:45.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.597+0000 7fa370910640 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55b9b363e960 0x55b9b3640d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.597+0000 7fa34bfff640 10 client.4965 ms_handle_connect on v2:172.21.15.145:6833/1058270486 2024-04-22T07:04:45.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.597+0000 7fa370910640 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55b9b363e960 0x55b9b3640d50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2bea213300ab6216 server_cookie=2bfcaab80266319b in_seq=0 out_seq=0 2024-04-22T07:04:45.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa34bfff640 1 -- 172.21.15.92:0/3267982273 <== mds.4 v2:172.21.15.145:6833/1058270486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9b354e530 con 0x55b9b363e960 2024-04-22T07:04:45.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa34bfff640 10 client.4965 handle_command_reply: tid=0 2024-04-22T07:04:45.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa375bfd740 1 client.4965 shutdown 2024-04-22T07:04:45.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa375bfd740 1 -- 172.21.15.92:0/3267982273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 msgr2=0x55b9b3605910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa375bfd740 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b3605910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa368008010 tx=0x7fa368004730 comp rx=0 tx=0).stop 2024-04-22T07:04:45.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa375bfd740 1 -- 172.21.15.92:0/3267982273 shutdown_connections 2024-04-22T07:04:45.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa375bfd740 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55b9b363e960 0x55b9b3640d50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa375bfd740 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b36019e0 0x55b9b36234f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa375bfd740 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361d220 0x55b9b36222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa34bfff640 0 client.4965 ms_handle_reset on v2:172.21.15.145:6833/1058270486 2024-04-22T07:04:45.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa375bfd740 1 --2- 172.21.15.92:0/3267982273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b3605910 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa375bfd740 1 -- 172.21.15.92:0/3267982273 >> 172.21.15.92:0/3267982273 conn(0x55b9b35ddec0 msgr2=0x55b9b35d6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa375bfd740 1 -- 172.21.15.92:0/3267982273 shutdown_connections 2024-04-22T07:04:45.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.598+0000 7fa375bfd740 1 -- 172.21.15.92:0/3267982273 wait complete. 2024-04-22T07:04:45.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.599+0000 7fa375bfd740 20 client.4965 trim_cache size 0 max 16384 2024-04-22T07:04:45.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.617+0000 7fa375bfd740 1 Processor -- start 2024-04-22T07:04:45.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.617+0000 7fa375bfd740 1 -- start start 2024-04-22T07:04:45.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.617+0000 7fa375bfd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b36019e0 0x55b9b3631870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.617+0000 7fa375bfd740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361ce40 0x55b9b3626e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.617+0000 7fa375bfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361d220 0x55b9b3627340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.617+0000 7fa375bfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1f23f0 con 0x55b9b361d220 2024-04-22T07:04:45.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.617+0000 7fa375bfd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1f2ac0 con 0x55b9b36019e0 2024-04-22T07:04:45.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.617+0000 7fa375bfd740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55b9b36014c0 con 0x55b9b361ce40 2024-04-22T07:04:45.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.617+0000 7fa371912640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361d220 0x55b9b3627340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.617+0000 7fa371912640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361d220 0x55b9b3627340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51702/0 (socket says 172.21.15.92:51702) 2024-04-22T07:04:45.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.617+0000 7fa371912640 1 -- 172.21.15.92:0/4260053951 learned_addr learned my addr 172.21.15.92:0/4260053951 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa371111640 1 --2- 172.21.15.92:0/4260053951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b36019e0 0x55b9b3631870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa370910640 1 --2- 172.21.15.92:0/4260053951 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361ce40 0x55b9b3626e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa371912640 1 -- 172.21.15.92:0/4260053951 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361ce40 msgr2=0x55b9b3626e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa371912640 1 --2- 172.21.15.92:0/4260053951 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361ce40 0x55b9b3626e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa371912640 1 -- 172.21.15.92:0/4260053951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b36019e0 msgr2=0x55b9b3631870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa371912640 1 --2- 172.21.15.92:0/4260053951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b36019e0 0x55b9b3631870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa371912640 1 -- 172.21.15.92:0/4260053951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b3622620 con 0x55b9b361d220 2024-04-22T07:04:45.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa371912640 1 --2- 172.21.15.92:0/4260053951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361d220 0x55b9b3627340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa3640849c0 tx=0x7fa36407c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389e67e46cf4b2ed server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa3567fc640 1 -- 172.21.15.92:0/4260053951 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364080630 con 0x55b9b361d220 2024-04-22T07:04:45.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa3567fc640 1 -- 172.21.15.92:0/4260053951 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa364084ef0 con 0x55b9b361d220 2024-04-22T07:04:45.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa3567fc640 1 -- 172.21.15.92:0/4260053951 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36407a580 con 0x55b9b361d220 2024-04-22T07:04:45.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa375bfd740 1 -- 172.21.15.92:0/4260053951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361d220 msgr2=0x55b9b3627340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.618+0000 7fa375bfd740 1 --2- 172.21.15.92:0/4260053951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361d220 0x55b9b3627340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa3640849c0 tx=0x7fa36407c180 comp rx=0 tx=0).stop 2024-04-22T07:04:45.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.619+0000 7fa375bfd740 1 -- 172.21.15.92:0/4260053951 shutdown_connections 2024-04-22T07:04:45.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.619+0000 7fa375bfd740 1 --2- 172.21.15.92:0/4260053951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361d220 0x55b9b3627340 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.619+0000 7fa375bfd740 1 --2- 172.21.15.92:0/4260053951 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b361ce40 0x55b9b3626e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.619+0000 7fa375bfd740 1 --2- 172.21.15.92:0/4260053951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b36019e0 0x55b9b3631870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.619+0000 7fa375bfd740 1 -- 172.21.15.92:0/4260053951 >> 172.21.15.92:0/4260053951 conn(0x55b9b35ddec0 msgr2=0x55b9b35dc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.619+0000 7fa375bfd740 1 -- 172.21.15.92:0/4260053951 shutdown_connections 2024-04-22T07:04:45.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.619+0000 7fa375bfd740 1 -- 172.21.15.92:0/4260053951 wait complete. 2024-04-22T07:04:45.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.619+0000 7fa375bfd740 1 -- start start 2024-04-22T07:04:45.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.619+0000 7fa375bfd740 1 Processor -- start 2024-04-22T07:04:45.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa375bfd740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b36019e0 0x55b9b3696260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa375bfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b3620dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa375bfd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b361d220 0x55b9b35ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa375bfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b9b3623a30 con 0x55b9b361ce40 2024-04-22T07:04:45.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa375bfd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b9b3602130 con 0x55b9b361d220 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa375bfd740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55b9b3635580 con 0x55b9b36019e0 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa375bfd740 10 client.0 _ll_register_callbacks cb 0x7fa368012d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa370910640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b3620dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa370910640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b3620dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51712/0 (socket says 172.21.15.92:51712) 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa370910640 1 -- 172.21.15.92:0/314209997 learned_addr learned my addr 172.21.15.92:0/314209997 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa371111640 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b36019e0 0x55b9b3696260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa371912640 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b361d220 0x55b9b35ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa370910640 1 -- 172.21.15.92:0/314209997 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b36019e0 msgr2=0x55b9b3696260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa370910640 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b36019e0 0x55b9b3696260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa370910640 1 -- 172.21.15.92:0/314209997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b361d220 msgr2=0x55b9b35ffb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa370910640 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b361d220 0x55b9b35ffb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.620+0000 7fa370910640 1 -- 172.21.15.92:0/314209997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b3622cb0 con 0x55b9b361ce40 2024-04-22T07:04:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa34bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa370910640 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b3620dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55b9b3602c50 tx=0x7fa35c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f005c9e4d8005389 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa34bfff640 1 -- 172.21.15.92:0/314209997 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c004240 con 0x55b9b361ce40 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa34bfff640 1 -- 172.21.15.92:0/314209997 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa35c0043e0 con 0x55b9b361ce40 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa375bfd740 1 -- 172.21.15.92:0/314209997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa364084ef0 con 0x55b9b361ce40 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa34bfff640 1 -- 172.21.15.92:0/314209997 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa35c027050 con 0x55b9b361ce40 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa375bfd740 10 client.4971 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa375bfd740 1 -- 172.21.15.92:0/314209997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b9b3621800 con 0x55b9b361ce40 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa34bfff640 1 -- 172.21.15.92:0/314209997 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa364084ef0 con 0x55b9b361ce40 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa34bfff640 1 -- 172.21.15.92:0/314209997 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fa35c02a020 con 0x55b9b361ce40 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa375bfd740 10 client.4971 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa375bfd740 10 client.4971 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa375bfd740 20 client.4971 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa375bfd740 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55b9b369fcf0 0x55b9b36a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa375bfd740 4 client.4971 mds_command: new command op to 4229 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:45.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa375bfd740 1 -- 172.21.15.92:0/314209997 --> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] -- command(tid 0: {"prefix": "status"}) -- 0x55b9b35483a0 con 0x55b9b369fcf0 2024-04-22T07:04:45.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.621+0000 7fa371111640 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55b9b369fcf0 0x55b9b36a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa34bfff640 10 client.4971 ms_handle_connect on v2:172.21.15.145:6833/1058270486 2024-04-22T07:04:45.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa371111640 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55b9b369fcf0 0x55b9b36a87e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7385bc7dd41eb52a server_cookie=1d31416ca4b8101b in_seq=0 out_seq=0 2024-04-22T07:04:45.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa34bfff640 1 -- 172.21.15.92:0/314209997 <== mds.4 v2:172.21.15.145:6833/1058270486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b9b35483a0 con 0x55b9b369fcf0 2024-04-22T07:04:45.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa34bfff640 10 client.4971 handle_command_reply: tid=0 2024-04-22T07:04:45.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa375bfd740 1 client.4971 shutdown 2024-04-22T07:04:45.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa375bfd740 1 -- 172.21.15.92:0/314209997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 msgr2=0x55b9b3620dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa375bfd740 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b3620dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55b9b3602c50 tx=0x7fa35c003c30 comp rx=0 tx=0).stop 2024-04-22T07:04:45.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa375bfd740 1 -- 172.21.15.92:0/314209997 shutdown_connections 2024-04-22T07:04:45.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa375bfd740 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.145:6833/1058270486,v1:172.21.15.145:6836/1058270486] conn(0x55b9b369fcf0 0x55b9b36a87e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa375bfd740 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b9b361d220 0x55b9b35ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa34bfff640 0 client.4971 ms_handle_reset on v2:172.21.15.145:6833/1058270486 2024-04-22T07:04:45.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa375bfd740 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9b361ce40 0x55b9b3620dd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa375bfd740 1 --2- 172.21.15.92:0/314209997 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55b9b36019e0 0x55b9b3696260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa375bfd740 1 -- 172.21.15.92:0/314209997 >> 172.21.15.92:0/314209997 conn(0x55b9b35ddec0 msgr2=0x55b9b35d5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa375bfd740 1 -- 172.21.15.92:0/314209997 shutdown_connections 2024-04-22T07:04:45.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa375bfd740 1 -- 172.21.15.92:0/314209997 wait complete. 2024-04-22T07:04:45.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.622+0000 7fa375bfd740 20 client.4971 trim_cache size 0 max 16384 2024-04-22T07:04:45.627 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:04:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "72eb29c5-e441-491c-9309-4fb4ef16ce27", 2024-04-22T07:04:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 4, 2024-04-22T07:04:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4229, 2024-04-22T07:04:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-04-22T07:04:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-04-22T07:04:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-04-22T07:04:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 20.598584705, 2024-04-22T07:04:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 20, 2024-04-22T07:04:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 20, 2024-04-22T07:04:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 48.704008590000001 2024-04-22T07:04:45.628 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:04:45.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.623+0000 7fa373375640 1 -- 172.21.15.92:0/1630413288 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa35006c8f0 msgr2=0x7fa35006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.623+0000 7fa373375640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa35006c8f0 0x7fa35006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa368006fd0 tx=0x7fa368008040 comp rx=0 tx=0).stop 2024-04-22T07:04:45.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.623+0000 7fa373375640 1 -- 172.21.15.92:0/1630413288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa36c158090 msgr2=0x7fa36c158d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.623+0000 7fa373375640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa36c158090 0x7fa36c158d50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa364036040 tx=0x7fa364004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:45.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.623+0000 7fa373375640 1 -- 172.21.15.92:0/1630413288 shutdown_connections 2024-04-22T07:04:45.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.623+0000 7fa373375640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fa35006c8f0 0x7fa35006edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.623+0000 7fa373375640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa36c158090 0x7fa36c158d50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.623+0000 7fa373375640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa36c151b70 0x7fa36c1ef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.623+0000 7fa373375640 1 --2- 172.21.15.92:0/1630413288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa36c0c7840 0x7fa36c154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.623+0000 7fa373375640 1 -- 172.21.15.92:0/1630413288 >> 172.21.15.92:0/1630413288 conn(0x7fa36c0cc220 msgr2=0x7fa36c0cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.623+0000 7fa373375640 1 -- 172.21.15.92:0/1630413288 shutdown_connections 2024-04-22T07:04:45.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.623+0000 7fa373375640 1 -- 172.21.15.92:0/1630413288 wait complete. 2024-04-22T07:04:45.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-22T07:04:45.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.730+0000 7fd311f84640 1 Processor -- start 2024-04-22T07:04:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.731+0000 7fd311f84640 1 -- start start 2024-04-22T07:04:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.731+0000 7fd311f84640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd30c067920 0x7fd30c15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.731+0000 7fd311f84640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd30c0c76d0 0x7fd30c154b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.731+0000 7fd311f84640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd30c0c7e40 con 0x7fd30c067d00 2024-04-22T07:04:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.731+0000 7fd311f84640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd30c06d370 con 0x7fd30c0c76d0 2024-04-22T07:04:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.731+0000 7fd311f84640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd30c0c8100 con 0x7fd30c067920 2024-04-22T07:04:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.732+0000 7fd30affd640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd30c0c76d0 0x7fd30c154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.732+0000 7fd30b7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd30c067920 0x7fd30c15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.732+0000 7fd30b7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd30c067920 0x7fd30c15e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.92:45110/0 (socket says 172.21.15.92:45110) 2024-04-22T07:04:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.732+0000 7fd30affd640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd30c0c76d0 0x7fd30c154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:51436/0 (socket says 172.21.15.92:51436) 2024-04-22T07:04:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.732+0000 7fd30b7fe640 1 -- 172.21.15.92:0/836936155 learned_addr learned my addr 172.21.15.92:0/836936155 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.732+0000 7fd30bfff640 1 --1- 172.21.15.92:0/836936155 >> v1:172.21.15.145:6789/0 conn(0x7fd30c067d00 0x7fd30c15ad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.92:38586/0 (socket says 172.21.15.92:38586) 2024-04-22T07:04:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.732+0000 7fd30b7fe640 1 -- 172.21.15.92:0/836936155 >> v2:172.21.15.145:3300/0 conn(0x7fd30c0c76d0 msgr2=0x7fd30c154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.732+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/836936155 >> v2:172.21.15.145:3300/0 conn(0x7fd30c0c76d0 0x7fd30c154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.732+0000 7fd30b7fe640 1 -- 172.21.15.92:0/836936155 >> v1:172.21.15.145:6789/0 conn(0x7fd30c067d00 legacy=0x7fd30c15ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.732+0000 7fd30b7fe640 1 -- 172.21.15.92:0/836936155 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30c06c890 con 0x7fd30c067920 2024-04-22T07:04:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.733+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/836936155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd30c067920 0x7fd30c15e2a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd300009870 tx=0x7fd30002ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d84b13a670c54ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.733+0000 7fd30a7fc640 1 -- 172.21.15.92:0/836936155 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30002f840 con 0x7fd30c067920 2024-04-22T07:04:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.733+0000 7fd30a7fc640 1 -- 172.21.15.92:0/836936155 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30002f9e0 con 0x7fd30c067920 2024-04-22T07:04:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.733+0000 7fd30a7fc640 1 -- 172.21.15.92:0/836936155 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30002fdb0 con 0x7fd30c067920 2024-04-22T07:04:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.733+0000 7fd311f84640 1 -- 172.21.15.92:0/836936155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd30c067920 msgr2=0x7fd30c15e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.733+0000 7fd311f84640 1 --2- 172.21.15.92:0/836936155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd30c067920 0x7fd30c15e2a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd300009870 tx=0x7fd30002ba20 comp rx=0 tx=0).stop 2024-04-22T07:04:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.733+0000 7fd311f84640 1 -- 172.21.15.92:0/836936155 shutdown_connections 2024-04-22T07:04:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.733+0000 7fd311f84640 1 --2- 172.21.15.92:0/836936155 >> v2:172.21.15.145:3300/0 conn(0x7fd30c0c76d0 0x7fd30c154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.733+0000 7fd311f84640 1 --2- 172.21.15.92:0/836936155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd30c067920 0x7fd30c15e2a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.733+0000 7fd311f84640 1 -- 172.21.15.92:0/836936155 >> 172.21.15.92:0/836936155 conn(0x7fd30c0c9f40 msgr2=0x7fd30c0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.734+0000 7fd311f84640 1 -- 172.21.15.92:0/836936155 shutdown_connections 2024-04-22T07:04:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.734+0000 7fd311f84640 1 -- 172.21.15.92:0/836936155 wait complete. 2024-04-22T07:04:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.735+0000 7fd311f84640 1 Processor -- start 2024-04-22T07:04:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.735+0000 7fd311f84640 1 -- start start 2024-04-22T07:04:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.735+0000 7fd311f84640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd30c067920 0x7fd30c151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.735+0000 7fd311f84640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd30c067d00 0x7fd30c153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.735+0000 7fd311f84640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd30c0c76d0 0x7fd30c153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.735+0000 7fd311f84640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c06b970 con 0x7fd30c067d00 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.735+0000 7fd311f84640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c06ce50 con 0x7fd30c0c76d0 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.735+0000 7fd311f84640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd30c06baf0 con 0x7fd30c067920 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd30affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd30c067d00 0x7fd30c153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd30affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd30c067d00 0x7fd30c153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51722/0 (socket says 172.21.15.92:51722) 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd30affd640 1 -- 172.21.15.92:0/3171774858 learned_addr learned my addr 172.21.15.92:0/3171774858 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd30c067920 0x7fd30c151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd30bfff640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd30c0c76d0 0x7fd30c153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd30affd640 1 -- 172.21.15.92:0/3171774858 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd30c067920 msgr2=0x7fd30c151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd30affd640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd30c067920 0x7fd30c151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd30affd640 1 -- 172.21.15.92:0/3171774858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd30c0c76d0 msgr2=0x7fd30c153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd30affd640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd30c0c76d0 0x7fd30c153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd30affd640 1 -- 172.21.15.92:0/3171774858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30c1f0be0 con 0x7fd30c067d00 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd30affd640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd30c067d00 0x7fd30c153300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00e870 tx=0x7fd2fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd2f3fff640 1 -- 172.21.15.92:0/3171774858 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc017070 con 0x7fd30c067d00 2024-04-22T07:04:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd2f3fff640 1 -- 172.21.15.92:0/3171774858 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2fc00c940 con 0x7fd30c067d00 2024-04-22T07:04:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd311f84640 1 -- 172.21.15.92:0/3171774858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30c1f0ec0 con 0x7fd30c067d00 2024-04-22T07:04:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd2f3fff640 1 -- 172.21.15.92:0/3171774858 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00cd10 con 0x7fd30c067d00 2024-04-22T07:04:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.736+0000 7fd311f84640 1 -- 172.21.15.92:0/3171774858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd30c1f1480 con 0x7fd30c067d00 2024-04-22T07:04:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.738+0000 7fd2f3fff640 1 -- 172.21.15.92:0/3171774858 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2fc01a070 con 0x7fd30c067d00 2024-04-22T07:04:45.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.738+0000 7fd2f3fff640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fd2f806c8f0 0x7fd2f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.738+0000 7fd2f3fff640 1 -- 172.21.15.92:0/3171774858 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd2fc08bd50 con 0x7fd30c067d00 2024-04-22T07:04:45.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.738+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fd2f806c8f0 0x7fd2f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.739+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fd2f806c8f0 0x7fd2f806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd300036000 tx=0x7fd300008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.739+0000 7fd314b7b740 1 Processor -- start 2024-04-22T07:04:45.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.739+0000 7fd314b7b740 1 -- start start 2024-04-22T07:04:45.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd314b7b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 0x5625a19d8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd314b7b740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1990ec0 0x5625a19d94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd314b7b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a198cca0 0x5625a19b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd314b7b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5625a18f3b10 con 0x5625a19cfe40 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd314b7b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5625a1891240 con 0x5625a198cca0 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd314b7b740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5625a18efad0 con 0x5625a1990ec0 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd30b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 0x5625a19d8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd30b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 0x5625a19d8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51728/0 (socket says 172.21.15.92:51728) 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd30b7fe640 1 -- 172.21.15.92:0/4280200774 learned_addr learned my addr 172.21.15.92:0/4280200774 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd30bfff640 1 --2- 172.21.15.92:0/4280200774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a198cca0 0x5625a19b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd30affd640 1 --2- 172.21.15.92:0/4280200774 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1990ec0 0x5625a19d94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd30b7fe640 1 -- 172.21.15.92:0/4280200774 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1990ec0 msgr2=0x5625a19d94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/4280200774 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1990ec0 0x5625a19d94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd30b7fe640 1 -- 172.21.15.92:0/4280200774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a198cca0 msgr2=0x5625a19b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/4280200774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a198cca0 0x5625a19b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd30b7fe640 1 -- 172.21.15.92:0/4280200774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625a16062d0 con 0x5625a19cfe40 2024-04-22T07:04:45.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.740+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/4280200774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 0x5625a19d8f60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd3000090b0 tx=0x7fd300030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c86a7fe86e9bde server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.741+0000 7fd2f17fa640 1 -- 172.21.15.92:0/4280200774 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300053070 con 0x5625a19cfe40 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.741+0000 7fd2f17fa640 1 -- 172.21.15.92:0/4280200774 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd300043510 con 0x5625a19cfe40 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.741+0000 7fd2f17fa640 1 -- 172.21.15.92:0/4280200774 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3000438e0 con 0x5625a19cfe40 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.741+0000 7fd314b7b740 1 -- 172.21.15.92:0/4280200774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 msgr2=0x5625a19d8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.741+0000 7fd314b7b740 1 --2- 172.21.15.92:0/4280200774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 0x5625a19d8f60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd3000090b0 tx=0x7fd300030000 comp rx=0 tx=0).stop 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.741+0000 7fd314b7b740 1 -- 172.21.15.92:0/4280200774 shutdown_connections 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.741+0000 7fd314b7b740 1 --2- 172.21.15.92:0/4280200774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a198cca0 0x5625a19b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.741+0000 7fd314b7b740 1 --2- 172.21.15.92:0/4280200774 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1990ec0 0x5625a19d94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.741+0000 7fd314b7b740 1 --2- 172.21.15.92:0/4280200774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 0x5625a19d8f60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.741+0000 7fd314b7b740 1 -- 172.21.15.92:0/4280200774 >> 172.21.15.92:0/4280200774 conn(0x5625a198b990 msgr2=0x5625a198e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.741+0000 7fd314b7b740 1 -- 172.21.15.92:0/4280200774 shutdown_connections 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.741+0000 7fd314b7b740 1 -- 172.21.15.92:0/4280200774 wait complete. 2024-04-22T07:04:45.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.742+0000 7fd314b7b740 1 -- start start 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.742+0000 7fd314b7b740 1 Processor -- start 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd314b7b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1990ec0 0x5625a19b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd314b7b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a19cfe40 0x5625a19d7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd314b7b740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1988600 0x5625a19d44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd314b7b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5625a198f250 con 0x5625a1990ec0 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd314b7b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5625a187c190 con 0x5625a19cfe40 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd314b7b740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5625a198f530 con 0x5625a1988600 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd314b7b740 10 client.0 _ll_register_callbacks cb 0x5625a18778b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd30b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1990ec0 0x5625a19b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd30b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1990ec0 0x5625a19b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51736/0 (socket says 172.21.15.92:51736) 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd30b7fe640 1 -- 172.21.15.92:0/1210366382 learned_addr learned my addr 172.21.15.92:0/1210366382 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd30affd640 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a19cfe40 0x5625a19d7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd30bfff640 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1988600 0x5625a19d44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd30b7fe640 1 -- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1988600 msgr2=0x5625a19d44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1988600 0x5625a19d44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd30b7fe640 1 -- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a19cfe40 msgr2=0x5625a19d7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a19cfe40 0x5625a19d7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd30b7fe640 1 -- 172.21.15.92:0/1210366382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625a16d06b0 con 0x5625a1990ec0 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd30bfff640 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1988600 0x5625a19d44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1990ec0 0x5625a19b1510 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd3000090b0 tx=0x7fd300004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f45ec1f9220ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd2ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd2ea7fc640 1 -- 172.21.15.92:0/1210366382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30005e060 con 0x5625a1990ec0 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd2ea7fc640 1 -- 172.21.15.92:0/1210366382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd300059680 con 0x5625a1990ec0 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd314b7b740 1 -- 172.21.15.92:0/1210366382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd300043510 con 0x5625a1990ec0 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd2ea7fc640 1 -- 172.21.15.92:0/1210366382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd300064050 con 0x5625a1990ec0 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd314b7b740 10 client.4980 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.743+0000 7fd314b7b740 1 -- 172.21.15.92:0/1210366382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5625a19b1c30 con 0x5625a1990ec0 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.744+0000 7fd2ea7fc640 1 -- 172.21.15.92:0/1210366382 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd300043510 con 0x5625a1990ec0 2024-04-22T07:04:45.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.744+0000 7fd2ea7fc640 1 -- 172.21.15.92:0/1210366382 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fd300067020 con 0x5625a1990ec0 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.744+0000 7fd314b7b740 10 client.4980 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.744+0000 7fd314b7b740 10 client.4980 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.744+0000 7fd314b7b740 20 client.4980 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.744+0000 7fd314b7b740 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x5625a19ed370 0x5625a19ef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.744+0000 7fd314b7b740 4 client.4980 mds_command: new command op to 4232 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.744+0000 7fd314b7b740 1 -- 172.21.15.92:0/1210366382 --> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625a1901530 con 0x5625a19ed370 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.744+0000 7fd30affd640 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x5625a19ed370 0x5625a19ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.744+0000 7fd30affd640 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x5625a19ed370 0x5625a19ef760 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=22ef2da17d7d7c51 server_cookie=56a69964cca865ad in_seq=0 out_seq=0 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.744+0000 7fd2ea7fc640 10 client.4980 ms_handle_connect on v2:172.21.15.145:6834/3258302920 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd2ea7fc640 1 -- 172.21.15.92:0/1210366382 <== mds.3 v2:172.21.15.145:6834/3258302920 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5625a1901530 con 0x5625a19ed370 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd2ea7fc640 10 client.4980 handle_command_reply: tid=0 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd314b7b740 1 client.4980 shutdown 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd314b7b740 1 -- 172.21.15.92:0/1210366382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1990ec0 msgr2=0x5625a19b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd314b7b740 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1990ec0 0x5625a19b1510 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd3000090b0 tx=0x7fd300004730 comp rx=0 tx=0).stop 2024-04-22T07:04:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd314b7b740 1 -- 172.21.15.92:0/1210366382 shutdown_connections 2024-04-22T07:04:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd314b7b740 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1988600 0x5625a19d44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd314b7b740 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a19cfe40 0x5625a19d7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd314b7b740 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x5625a19ed370 0x5625a19ef760 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd314b7b740 1 --2- 172.21.15.92:0/1210366382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1990ec0 0x5625a19b1510 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd2ea7fc640 0 client.4980 ms_handle_reset on v2:172.21.15.145:6834/3258302920 2024-04-22T07:04:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd314b7b740 1 -- 172.21.15.92:0/1210366382 >> 172.21.15.92:0/1210366382 conn(0x5625a198cca0 msgr2=0x5625a19d0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd314b7b740 1 -- 172.21.15.92:0/1210366382 shutdown_connections 2024-04-22T07:04:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd314b7b740 1 -- 172.21.15.92:0/1210366382 wait complete. 2024-04-22T07:04:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.745+0000 7fd314b7b740 20 client.4980 trim_cache size 0 max 16384 2024-04-22T07:04:45.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.763+0000 7fd314b7b740 1 Processor -- start 2024-04-22T07:04:45.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.763+0000 7fd314b7b740 1 -- start start 2024-04-22T07:04:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd314b7b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1988600 0x5625a19d8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd314b7b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a1990ec0 0x5625a19d9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd314b7b740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a19cfe40 0x5625a19f2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd314b7b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c143930 con 0x5625a1988600 2024-04-22T07:04:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd314b7b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5625a1988170 con 0x5625a1990ec0 2024-04-22T07:04:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd314b7b740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5625a198a040 con 0x5625a19cfe40 2024-04-22T07:04:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd30b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1988600 0x5625a19d8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd30b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1988600 0x5625a19d8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51738/0 (socket says 172.21.15.92:51738) 2024-04-22T07:04:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd30b7fe640 1 -- 172.21.15.92:0/3245953172 learned_addr learned my addr 172.21.15.92:0/3245953172 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd30b7fe640 1 -- 172.21.15.92:0/3245953172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a19cfe40 msgr2=0x5625a19f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/3245953172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a19cfe40 0x5625a19f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd30b7fe640 1 -- 172.21.15.92:0/3245953172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a1990ec0 msgr2=0x5625a19d9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/3245953172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a1990ec0 0x5625a19d9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.764+0000 7fd30b7fe640 1 -- 172.21.15.92:0/3245953172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625a19902e0 con 0x5625a1988600 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/3245953172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1988600 0x5625a19d8c50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd300006ac0 tx=0x7fd300053000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c1751590d99aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd2f0ff9640 1 -- 172.21.15.92:0/3245953172 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300053070 con 0x5625a1988600 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd2f0ff9640 1 -- 172.21.15.92:0/3245953172 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3000439d0 con 0x5625a1988600 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd2f0ff9640 1 -- 172.21.15.92:0/3245953172 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300059680 con 0x5625a1988600 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd314b7b740 1 -- 172.21.15.92:0/3245953172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1988600 msgr2=0x5625a19d8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd314b7b740 1 --2- 172.21.15.92:0/3245953172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1988600 0x5625a19d8c50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd300006ac0 tx=0x7fd300053000 comp rx=0 tx=0).stop 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd314b7b740 1 -- 172.21.15.92:0/3245953172 shutdown_connections 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd314b7b740 1 --2- 172.21.15.92:0/3245953172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a19cfe40 0x5625a19f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd314b7b740 1 --2- 172.21.15.92:0/3245953172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a1990ec0 0x5625a19d9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd314b7b740 1 --2- 172.21.15.92:0/3245953172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a1988600 0x5625a19d8c50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd314b7b740 1 -- 172.21.15.92:0/3245953172 >> 172.21.15.92:0/3245953172 conn(0x5625a198cca0 msgr2=0x5625a1984800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd314b7b740 1 -- 172.21.15.92:0/3245953172 shutdown_connections 2024-04-22T07:04:45.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.765+0000 7fd314b7b740 1 -- 172.21.15.92:0/3245953172 wait complete. 2024-04-22T07:04:45.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.766+0000 7fd314b7b740 1 -- start start 2024-04-22T07:04:45.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.766+0000 7fd314b7b740 1 Processor -- start 2024-04-22T07:04:45.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd314b7b740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1988600 0x5625a19b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd314b7b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a1990ec0 0x5625a19d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd314b7b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 0x5625a19b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd314b7b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5625a1988b50 con 0x5625a19cfe40 2024-04-22T07:04:45.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd314b7b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5625a19b5630 con 0x5625a1990ec0 2024-04-22T07:04:45.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd314b7b740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5625a19c9640 con 0x5625a1988600 2024-04-22T07:04:45.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd314b7b740 10 client.0 _ll_register_callbacks cb 0x7fd300051070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:45.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd30bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 0x5625a19b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd30bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 0x5625a19b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51754/0 (socket says 172.21.15.92:51754) 2024-04-22T07:04:45.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd30bfff640 1 -- 172.21.15.92:0/285281694 learned_addr learned my addr 172.21.15.92:0/285281694 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd30affd640 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a1990ec0 0x5625a19d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1988600 0x5625a19b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd30bfff640 1 -- 172.21.15.92:0/285281694 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1988600 msgr2=0x5625a19b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd30bfff640 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1988600 0x5625a19b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd30bfff640 1 -- 172.21.15.92:0/285281694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a1990ec0 msgr2=0x5625a19d5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd30bfff640 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a1990ec0 0x5625a19d5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd30bfff640 1 -- 172.21.15.92:0/285281694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625a19d5980 con 0x5625a19cfe40 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd30bfff640 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 0x5625a19b2bc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd3040079e0 tx=0x7fd304012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e96b22ff4e73a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd2ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd2ea7fc640 1 -- 172.21.15.92:0/285281694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304016480 con 0x5625a19cfe40 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd2ea7fc640 1 -- 172.21.15.92:0/285281694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd304016620 con 0x5625a19cfe40 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.767+0000 7fd314b7b740 1 -- 172.21.15.92:0/285281694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3000439d0 con 0x5625a19cfe40 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd2ea7fc640 1 -- 172.21.15.92:0/285281694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd304023050 con 0x5625a19cfe40 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd314b7b740 10 client.4986 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd314b7b740 1 -- 172.21.15.92:0/285281694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5625a19908c0 con 0x5625a19cfe40 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd2ea7fc640 1 -- 172.21.15.92:0/285281694 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3000439d0 con 0x5625a19cfe40 2024-04-22T07:04:45.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd2ea7fc640 1 -- 172.21.15.92:0/285281694 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fd304026020 con 0x5625a19cfe40 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd314b7b740 10 client.4986 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd314b7b740 10 client.4986 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd314b7b740 20 client.4986 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd314b7b740 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x5625a1a54d60 0x5625a1a57150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd314b7b740 4 client.4986 mds_command: new command op to 4232 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd314b7b740 1 -- 172.21.15.92:0/285281694 --> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] -- command(tid 0: {"prefix": "status"}) -- 0x5625a18fb3a0 con 0x5625a1a54d60 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x5625a1a54d60 0x5625a1a57150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd2ea7fc640 10 client.4986 ms_handle_connect on v2:172.21.15.145:6834/3258302920 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.768+0000 7fd30b7fe640 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x5625a1a54d60 0x5625a1a57150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=42ad514ff6b6bc1e server_cookie=2c63868dc29ffe3b in_seq=0 out_seq=0 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd2ea7fc640 1 -- 172.21.15.92:0/285281694 <== mds.3 v2:172.21.15.145:6834/3258302920 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5625a18fb3a0 con 0x5625a1a54d60 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd2ea7fc640 10 client.4986 handle_command_reply: tid=0 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd314b7b740 1 client.4986 shutdown 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd314b7b740 1 -- 172.21.15.92:0/285281694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 msgr2=0x5625a19b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd314b7b740 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 0x5625a19b2bc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd3040079e0 tx=0x7fd304012630 comp rx=0 tx=0).stop 2024-04-22T07:04:45.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd314b7b740 1 -- 172.21.15.92:0/285281694 shutdown_connections 2024-04-22T07:04:45.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd314b7b740 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.145:6834/3258302920,v1:172.21.15.145:6837/3258302920] conn(0x5625a1a54d60 0x5625a1a57150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd314b7b740 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5625a19cfe40 0x5625a19b2bc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd314b7b740 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5625a1990ec0 0x5625a19d5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd314b7b740 1 --2- 172.21.15.92:0/285281694 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5625a1988600 0x5625a19b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd2ea7fc640 0 client.4986 ms_handle_reset on v2:172.21.15.145:6834/3258302920 2024-04-22T07:04:45.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd314b7b740 1 -- 172.21.15.92:0/285281694 >> 172.21.15.92:0/285281694 conn(0x5625a198cca0 msgr2=0x5625a19cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd314b7b740 1 -- 172.21.15.92:0/285281694 shutdown_connections 2024-04-22T07:04:45.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd314b7b740 1 -- 172.21.15.92:0/285281694 wait complete. 2024-04-22T07:04:45.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd314b7b740 20 client.4986 trim_cache size 0 max 16384 2024-04-22T07:04:45.773 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:04:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "72eb29c5-e441-491c-9309-4fb4ef16ce27", 2024-04-22T07:04:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 3, 2024-04-22T07:04:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4232, 2024-04-22T07:04:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-04-22T07:04:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-04-22T07:04:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-04-22T07:04:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 26.776884147000001, 2024-04-22T07:04:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 20, 2024-04-22T07:04:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 20, 2024-04-22T07:04:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 48.850231366999999 2024-04-22T07:04:45.774 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:04:45.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd311f84640 1 -- 172.21.15.92:0/3171774858 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fd2f806c8f0 msgr2=0x7fd2f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd311f84640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fd2f806c8f0 0x7fd2f806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd300036000 tx=0x7fd300008030 comp rx=0 tx=0).stop 2024-04-22T07:04:45.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd311f84640 1 -- 172.21.15.92:0/3171774858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd30c067d00 msgr2=0x7fd30c153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd311f84640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd30c067d00 0x7fd30c153300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00e870 tx=0x7fd2fc00ed30 comp rx=0 tx=0).stop 2024-04-22T07:04:45.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd311f84640 1 -- 172.21.15.92:0/3171774858 shutdown_connections 2024-04-22T07:04:45.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd311f84640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fd2f806c8f0 0x7fd2f806edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd311f84640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd30c0c76d0 0x7fd30c153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd311f84640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd30c067d00 0x7fd30c153300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd311f84640 1 --2- 172.21.15.92:0/3171774858 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd30c067920 0x7fd30c151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.769+0000 7fd311f84640 1 -- 172.21.15.92:0/3171774858 >> 172.21.15.92:0/3171774858 conn(0x7fd30c0c9f40 msgr2=0x7fd30c0cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.770+0000 7fd311f84640 1 -- 172.21.15.92:0/3171774858 shutdown_connections 2024-04-22T07:04:45.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.770+0000 7fd311f84640 1 -- 172.21.15.92:0/3171774858 wait complete. 2024-04-22T07:04:45.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-22T07:04:45.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.878+0000 7f0c206e0640 1 Processor -- start 2024-04-22T07:04:45.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.878+0000 7f0c206e0640 1 -- start start 2024-04-22T07:04:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c206e0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0c180c87e0 0x7f0c1806fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c206e0640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c18158de0 0x7f0c181591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c206e0640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c180c72b0 con 0x7f0c18152d80 2024-04-22T07:04:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c206e0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f0c1806ce60 con 0x7f0c180c87e0 2024-04-22T07:04:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c206e0640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0c18153160 con 0x7f0c18158de0 2024-04-22T07:04:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c1dc7b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0c180c87e0 0x7f0c1806fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c1e47c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0c18152d80 0x7f0c180ca020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48384/0 (socket says 172.21.15.92:48384) 2024-04-22T07:04:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c1e47c640 1 -- 172.21.15.92:0/1270750096 learned_addr learned my addr 172.21.15.92:0/1270750096 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:04:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c1ec7d640 1 --2- 172.21.15.92:0/1270750096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c18158de0 0x7f0c181591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/1270750096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c18158de0 msgr2=0x7f0c181591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/1270750096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c18158de0 0x7f0c181591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/1270750096 >> v1:172.21.15.92:6789/0 conn(0x7f0c18152d80 legacy=0x7f0c180ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/1270750096 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c180c6370 con 0x7f0c180c87e0 2024-04-22T07:04:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.879+0000 7f0c1ec7d640 1 --2- 172.21.15.92:0/1270750096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c18158de0 0x7f0c181591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:45.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.880+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/1270750096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c180c87e0 0x7f0c1806fec0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0c080097a0 tx=0x7f0c0802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4cdf0b6e5d142f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.880+0000 7f0c1d47a640 1 -- 172.21.15.92:0/1270750096 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08034070 con 0x7f0c180c87e0 2024-04-22T07:04:45.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.880+0000 7f0c1d47a640 1 -- 172.21.15.92:0/1270750096 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c08030610 con 0x7f0c180c87e0 2024-04-22T07:04:45.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.880+0000 7f0c1d47a640 1 -- 172.21.15.92:0/1270750096 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c080309e0 con 0x7f0c180c87e0 2024-04-22T07:04:45.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.881+0000 7f0c206e0640 1 -- 172.21.15.92:0/1270750096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c180c87e0 msgr2=0x7f0c1806fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.881+0000 7f0c206e0640 1 --2- 172.21.15.92:0/1270750096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c180c87e0 0x7f0c1806fec0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0c080097a0 tx=0x7f0c0802b960 comp rx=0 tx=0).stop 2024-04-22T07:04:45.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.881+0000 7f0c206e0640 1 -- 172.21.15.92:0/1270750096 shutdown_connections 2024-04-22T07:04:45.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.881+0000 7f0c206e0640 1 --2- 172.21.15.92:0/1270750096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c18158de0 0x7f0c181591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.881+0000 7f0c206e0640 1 --2- 172.21.15.92:0/1270750096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c180c87e0 0x7f0c1806fec0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.881+0000 7f0c206e0640 1 -- 172.21.15.92:0/1270750096 >> 172.21.15.92:0/1270750096 conn(0x7f0c180cc220 msgr2=0x7f0c180cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.881+0000 7f0c206e0640 1 -- 172.21.15.92:0/1270750096 shutdown_connections 2024-04-22T07:04:45.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.881+0000 7f0c206e0640 1 -- 172.21.15.92:0/1270750096 wait complete. 2024-04-22T07:04:45.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.882+0000 7f0c206e0640 1 Processor -- start 2024-04-22T07:04:45.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.882+0000 7f0c206e0640 1 -- start start 2024-04-22T07:04:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c206e0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c180c87e0 0x7f0c18154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c206e0640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c18152d80 0x7f0c181554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c206e0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c18158de0 0x7f0c1815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c206e0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0c180c5db0 con 0x7f0c180c87e0 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c206e0640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0c180c5ad0 con 0x7f0c18158de0 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c206e0640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0c180c6090 con 0x7f0c18152d80 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c1e47c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c180c87e0 0x7f0c18154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c1e47c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c180c87e0 0x7f0c18154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51774/0 (socket says 172.21.15.92:51774) 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c1e47c640 1 -- 172.21.15.92:0/2590378022 learned_addr learned my addr 172.21.15.92:0/2590378022 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c18152d80 0x7f0c181554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c1ec7d640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c18158de0 0x7f0c1815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c1e47c640 1 -- 172.21.15.92:0/2590378022 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c18152d80 msgr2=0x7f0c181554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c18152d80 0x7f0c181554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c1e47c640 1 -- 172.21.15.92:0/2590378022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c18158de0 msgr2=0x7f0c1815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c18158de0 0x7f0c1815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c1e47c640 1 -- 172.21.15.92:0/2590378022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1815ca70 con 0x7f0c180c87e0 2024-04-22T07:04:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c1ec7d640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c18158de0 0x7f0c1815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c180c87e0 0x7f0c18154fa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400cb40 tx=0x7f0c1400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c0effd640 1 -- 172.21.15.92:0/2590378022 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14017070 con 0x7f0c180c87e0 2024-04-22T07:04:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c0effd640 1 -- 172.21.15.92:0/2590378022 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1400aca0 con 0x7f0c180c87e0 2024-04-22T07:04:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c206e0640 1 -- 172.21.15.92:0/2590378022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c18156140 con 0x7f0c180c87e0 2024-04-22T07:04:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c0effd640 1 -- 172.21.15.92:0/2590378022 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14012620 con 0x7f0c180c87e0 2024-04-22T07:04:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.883+0000 7f0c206e0640 1 -- 172.21.15.92:0/2590378022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c181ef860 con 0x7f0c180c87e0 2024-04-22T07:04:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.885+0000 7f0c0effd640 1 -- 172.21.15.92:0/2590378022 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c1401a070 con 0x7f0c180c87e0 2024-04-22T07:04:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.886+0000 7f0c0effd640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0bfc06c8f0 0x7f0bfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.886+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0bfc06c8f0 0x7f0bfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.886+0000 7f0c0effd640 1 -- 172.21.15.92:0/2590378022 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c1408b930 con 0x7f0c180c87e0 2024-04-22T07:04:45.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.886+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0bfc06c8f0 0x7f0bfc06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c08036000 tx=0x7f0c0803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.886+0000 7f0c233ae740 1 Processor -- start 2024-04-22T07:04:45.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.887+0000 7f0c233ae740 1 -- start start 2024-04-22T07:04:45.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.887+0000 7f0c233ae740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb27e40 0x55bd3bae71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.887+0000 7f0c233ae740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bae8ec0 0x55bd3bb0cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.887+0000 7f0c233ae740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb28220 0x55bd3bb0d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.887+0000 7f0c233ae740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bd3ba4bb10 con 0x55bd3bae8ec0 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.887+0000 7f0c233ae740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bd3b9e9240 con 0x55bd3bb27e40 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.887+0000 7f0c233ae740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55bd3ba47ad0 con 0x55bd3bb28220 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.887+0000 7f0c1dc7b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bae8ec0 0x55bd3bb0cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.887+0000 7f0c1dc7b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bae8ec0 0x55bd3bb0cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51778/0 (socket says 172.21.15.92:51778) 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.887+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/2208129987 learned_addr learned my addr 172.21.15.92:0/2208129987 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.887+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/2208129987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb27e40 0x55bd3bae71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.888+0000 7f0c1ec7d640 1 --2- 172.21.15.92:0/2208129987 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb28220 0x55bd3bb0d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.888+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/2208129987 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb28220 msgr2=0x55bd3bb0d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.888+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/2208129987 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb28220 0x55bd3bb0d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.888+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/2208129987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb27e40 msgr2=0x55bd3bae71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.888+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/2208129987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb27e40 0x55bd3bae71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.888+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/2208129987 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd3b75e2d0 con 0x55bd3bae8ec0 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.888+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/2208129987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bae8ec0 0x55bd3bb0cd50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009890 tx=0x7f0c08030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd607ce31de6267 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.888+0000 7f0bfbfff640 1 -- 172.21.15.92:0/2208129987 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08034040 con 0x55bd3bae8ec0 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.888+0000 7f0bfbfff640 1 -- 172.21.15.92:0/2208129987 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c08054070 con 0x55bd3bae8ec0 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.888+0000 7f0bfbfff640 1 -- 172.21.15.92:0/2208129987 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08045dc0 con 0x55bd3bae8ec0 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.889+0000 7f0c233ae740 1 -- 172.21.15.92:0/2208129987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bae8ec0 msgr2=0x55bd3bb0cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.889+0000 7f0c233ae740 1 --2- 172.21.15.92:0/2208129987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bae8ec0 0x55bd3bb0cd50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009890 tx=0x7f0c08030000 comp rx=0 tx=0).stop 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.889+0000 7f0c233ae740 1 -- 172.21.15.92:0/2208129987 shutdown_connections 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.889+0000 7f0c233ae740 1 --2- 172.21.15.92:0/2208129987 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb28220 0x55bd3bb0d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.889+0000 7f0c233ae740 1 --2- 172.21.15.92:0/2208129987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bae8ec0 0x55bd3bb0cd50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.889+0000 7f0c233ae740 1 --2- 172.21.15.92:0/2208129987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb27e40 0x55bd3bae71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.889+0000 7f0c233ae740 1 -- 172.21.15.92:0/2208129987 >> 172.21.15.92:0/2208129987 conn(0x55bd3bae3990 msgr2=0x55bd3bae6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.889+0000 7f0c233ae740 1 -- 172.21.15.92:0/2208129987 shutdown_connections 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.889+0000 7f0c233ae740 1 -- 172.21.15.92:0/2208129987 wait complete. 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.889+0000 7f0c233ae740 1 -- start start 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.889+0000 7f0c233ae740 1 Processor -- start 2024-04-22T07:04:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c233ae740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb27e40 0x55bd3bb0bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c233ae740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb28220 0x55bd3bb3d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c233ae740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb0e600 0x55bd3bb079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c233ae740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bd3bae2220 con 0x55bd3bb28220 2024-04-22T07:04:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c233ae740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bd3b9d4190 con 0x55bd3bb0e600 2024-04-22T07:04:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c233ae740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55bd3bae1540 con 0x55bd3bb27e40 2024-04-22T07:04:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c233ae740 10 client.0 _ll_register_callbacks cb 0x55bd3b9cf8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c1dc7b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb28220 0x55bd3bb3d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c1dc7b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb28220 0x55bd3bb3d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51794/0 (socket says 172.21.15.92:51794) 2024-04-22T07:04:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/18969945 learned_addr learned my addr 172.21.15.92:0/18969945 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb27e40 0x55bd3bb0bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c1ec7d640 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb0e600 0x55bd3bb079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/18969945 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb27e40 msgr2=0x55bd3bb0bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb27e40 0x55bd3bb0bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/18969945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb0e600 msgr2=0x55bd3bb079f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb0e600 0x55bd3bb079f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/18969945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd3b8286b0 con 0x55bd3bb28220 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.890+0000 7f0c1ec7d640 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb0e600 0x55bd3bb079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb28220 0x55bd3bb3d2d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009560 tx=0x7f0c08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965d3fee2eadec39 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0bf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0bf97fa640 1 -- 172.21.15.92:0/18969945 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0805e070 con 0x55bd3bb28220 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0bf97fa640 1 -- 172.21.15.92:0/18969945 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c0805a910 con 0x55bd3bb28220 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0c233ae740 1 -- 172.21.15.92:0/18969945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c08054070 con 0x55bd3bb28220 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0bf97fa640 1 -- 172.21.15.92:0/18969945 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0c08069050 con 0x55bd3bb28220 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0c233ae740 10 client.5001 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:45.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0c233ae740 1 -- 172.21.15.92:0/18969945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55bd3bb29500 con 0x55bd3bb28220 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0bf97fa640 1 -- 172.21.15.92:0/18969945 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c08054070 con 0x55bd3bb28220 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0bf97fa640 1 -- 172.21.15.92:0/18969945 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f0c0806c020 con 0x55bd3bb28220 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0c233ae740 10 client.5001 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0c233ae740 10 client.5001 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0c233ae740 20 client.5001 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0c233ae740 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55bd3bb40100 0x55bd3bb4b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0c233ae740 4 client.5001 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0c233ae740 1 -- 172.21.15.92:0/18969945 --> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd3ba59530 con 0x55bd3bb40100 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.891+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55bd3bb40100 0x55bd3bb4b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.892+0000 7f0bf97fa640 10 client.5001 ms_handle_connect on v2:172.21.15.92:6835/1444537065 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.892+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55bd3bb40100 0x55bd3bb4b9b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92318306185e533c server_cookie=439aa8a42f01f759 in_seq=0 out_seq=0 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.892+0000 7f0bf97fa640 1 -- 172.21.15.92:0/18969945 <== mds.2 v2:172.21.15.92:6835/1444537065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd3ba59530 con 0x55bd3bb40100 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.892+0000 7f0bf97fa640 10 client.5001 handle_command_reply: tid=0 2024-04-22T07:04:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.892+0000 7f0c233ae740 1 client.5001 shutdown 2024-04-22T07:04:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.892+0000 7f0c233ae740 1 -- 172.21.15.92:0/18969945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb28220 msgr2=0x55bd3bb3d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.892+0000 7f0c233ae740 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb28220 0x55bd3bb3d2d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009560 tx=0x7f0c08004730 comp rx=0 tx=0).stop 2024-04-22T07:04:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.893+0000 7f0c233ae740 1 -- 172.21.15.92:0/18969945 shutdown_connections 2024-04-22T07:04:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.893+0000 7f0c233ae740 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55bd3bb40100 0x55bd3bb4b9b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.893+0000 7f0c233ae740 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb0e600 0x55bd3bb079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.893+0000 7f0c233ae740 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb28220 0x55bd3bb3d2d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.893+0000 7f0bf97fa640 0 client.5001 ms_handle_reset on v2:172.21.15.92:6835/1444537065 2024-04-22T07:04:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.893+0000 7f0c233ae740 1 --2- 172.21.15.92:0/18969945 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb27e40 0x55bd3bb0bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.893+0000 7f0c233ae740 1 -- 172.21.15.92:0/18969945 >> 172.21.15.92:0/18969945 conn(0x55bd3bae8ec0 msgr2=0x55bd3bae7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.893+0000 7f0c233ae740 1 -- 172.21.15.92:0/18969945 shutdown_connections 2024-04-22T07:04:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.893+0000 7f0c233ae740 1 -- 172.21.15.92:0/18969945 wait complete. 2024-04-22T07:04:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.893+0000 7f0c233ae740 20 client.5001 trim_cache size 0 max 16384 2024-04-22T07:04:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.910+0000 7f0c233ae740 1 Processor -- start 2024-04-22T07:04:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.910+0000 7f0c233ae740 1 -- start start 2024-04-22T07:04:45.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c233ae740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb0e600 0x55bd3bb254c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c233ae740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb27e40 0x55bd3bb25a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c233ae740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb28220 0x55bd3bb0ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c233ae740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1814d590 con 0x55bd3bb0e600 2024-04-22T07:04:45.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c233ae740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bd3bb08400 con 0x55bd3bb28220 2024-04-22T07:04:45.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c233ae740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55bd3bb29ac0 con 0x55bd3bb27e40 2024-04-22T07:04:45.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c1e47c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb0e600 0x55bd3bb254c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c1e47c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb0e600 0x55bd3bb254c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51800/0 (socket says 172.21.15.92:51800) 2024-04-22T07:04:45.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c1e47c640 1 -- 172.21.15.92:0/3443058730 learned_addr learned my addr 172.21.15.92:0/3443058730 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/3443058730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb27e40 0x55bd3bb25a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c1e47c640 1 -- 172.21.15.92:0/3443058730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb27e40 msgr2=0x55bd3bb25a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c1ec7d640 1 --2- 172.21.15.92:0/3443058730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb28220 0x55bd3bb0ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/3443058730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb27e40 0x55bd3bb25a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c1e47c640 1 -- 172.21.15.92:0/3443058730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb28220 msgr2=0x55bd3bb0ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/3443058730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb28220 0x55bd3bb0ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.911+0000 7f0c1e47c640 1 -- 172.21.15.92:0/3443058730 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd3bb2a4d0 con 0x55bd3bb0e600 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/3443058730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb27e40 0x55bd3bb25a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0c1ec7d640 1 --2- 172.21.15.92:0/3443058730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb28220 0x55bd3bb0ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/3443058730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb0e600 0x55bd3bb254c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0c14001330 tx=0x7f0c14089d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c801f484d0b8358 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0bfb7fe640 1 -- 172.21.15.92:0/3443058730 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14057700 con 0x55bd3bb0e600 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0bfb7fe640 1 -- 172.21.15.92:0/3443058730 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c14076070 con 0x55bd3bb0e600 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0bfb7fe640 1 -- 172.21.15.92:0/3443058730 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14055cd0 con 0x55bd3bb0e600 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0c233ae740 1 -- 172.21.15.92:0/3443058730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb0e600 msgr2=0x55bd3bb254c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0c233ae740 1 --2- 172.21.15.92:0/3443058730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb0e600 0x55bd3bb254c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0c14001330 tx=0x7f0c14089d30 comp rx=0 tx=0).stop 2024-04-22T07:04:45.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0c233ae740 1 -- 172.21.15.92:0/3443058730 shutdown_connections 2024-04-22T07:04:45.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0c233ae740 1 --2- 172.21.15.92:0/3443058730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb28220 0x55bd3bb0ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0c233ae740 1 --2- 172.21.15.92:0/3443058730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb27e40 0x55bd3bb25a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0c233ae740 1 --2- 172.21.15.92:0/3443058730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb0e600 0x55bd3bb254c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0c233ae740 1 -- 172.21.15.92:0/3443058730 >> 172.21.15.92:0/3443058730 conn(0x55bd3bae8ec0 msgr2=0x55bd3badc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0c233ae740 1 -- 172.21.15.92:0/3443058730 shutdown_connections 2024-04-22T07:04:45.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.912+0000 7f0c233ae740 1 -- 172.21.15.92:0/3443058730 wait complete. 2024-04-22T07:04:45.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.913+0000 7f0c233ae740 1 -- start start 2024-04-22T07:04:45.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.913+0000 7f0c233ae740 1 Processor -- start 2024-04-22T07:04:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c233ae740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb0e600 0x55bd3bba6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c233ae740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb27e40 0x55bd3bb255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c233ae740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb28220 0x55bd3bba0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c233ae740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bd3bb3ff80 con 0x55bd3bb27e40 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c233ae740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bd3bb29fe0 con 0x55bd3bb0e600 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c233ae740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55bd3bb40630 con 0x55bd3bb28220 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c233ae740 10 client.0 _ll_register_callbacks cb 0x7f0c08039910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c1dc7b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb27e40 0x55bd3bb255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c1dc7b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb27e40 0x55bd3bb255e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51804/0 (socket says 172.21.15.92:51804) 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/3760793519 learned_addr learned my addr 172.21.15.92:0/3760793519 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb0e600 0x55bd3bba6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c1ec7d640 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb28220 0x55bd3bba0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/3760793519 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb28220 msgr2=0x55bd3bba0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb28220 0x55bd3bba0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/3760793519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb0e600 msgr2=0x55bd3bba6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb0e600 0x55bd3bba6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c1dc7b640 1 -- 172.21.15.92:0/3760793519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd3bb51770 con 0x55bd3bb27e40 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb0e600 0x55bd3bba6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c1dc7b640 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb27e40 0x55bd3bb255e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c0802b9a0 tx=0x7f0c08008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e67c6a2e4c3b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0bf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0bf97fa640 1 -- 172.21.15.92:0/3760793519 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0805e070 con 0x55bd3bb27e40 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0bf97fa640 1 -- 172.21.15.92:0/3760793519 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c08064cf0 con 0x55bd3bb27e40 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.914+0000 7f0c233ae740 1 -- 172.21.15.92:0/3760793519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c14076070 con 0x55bd3bb27e40 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0bf97fa640 1 -- 172.21.15.92:0/3760793519 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0c08073050 con 0x55bd3bb27e40 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0c233ae740 10 client.5007 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0c233ae740 1 -- 172.21.15.92:0/3760793519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55bd3bb50f70 con 0x55bd3bb27e40 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0bf97fa640 1 -- 172.21.15.92:0/3760793519 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c14076070 con 0x55bd3bb27e40 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0bf97fa640 1 -- 172.21.15.92:0/3760793519 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f0c08076020 con 0x55bd3bb27e40 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0c233ae740 10 client.5007 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:45.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0c233ae740 10 client.5007 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0c233ae740 20 client.5007 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0c233ae740 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55bd3bbaa960 0x55bd3bbb3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0c233ae740 4 client.5007 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0c233ae740 1 -- 172.21.15.92:0/3760793519 --> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] -- command(tid 0: {"prefix": "status"}) -- 0x55bd3ba533a0 con 0x55bd3bbaa960 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55bd3bbaa960 0x55bd3bbb3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0bf97fa640 10 client.5007 ms_handle_connect on v2:172.21.15.92:6835/1444537065 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0c1e47c640 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55bd3bbaa960 0x55bd3bbb3400 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50578dfe86f9a013 server_cookie=79277d96bab0572b in_seq=0 out_seq=0 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0bf97fa640 1 -- 172.21.15.92:0/3760793519 <== mds.2 v2:172.21.15.92:6835/1444537065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55bd3ba533a0 con 0x55bd3bbaa960 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0bf97fa640 10 client.5007 handle_command_reply: tid=0 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.915+0000 7f0c233ae740 1 client.5007 shutdown 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c233ae740 1 -- 172.21.15.92:0/3760793519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb27e40 msgr2=0x55bd3bb255e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c233ae740 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb27e40 0x55bd3bb255e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c0802b9a0 tx=0x7f0c08008030 comp rx=0 tx=0).stop 2024-04-22T07:04:45.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c233ae740 1 -- 172.21.15.92:0/3760793519 shutdown_connections 2024-04-22T07:04:45.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c233ae740 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.92:6835/1444537065,v1:172.21.15.92:6836/1444537065] conn(0x55bd3bbaa960 0x55bd3bbb3400 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c233ae740 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55bd3bb28220 0x55bd3bba0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c233ae740 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd3bb27e40 0x55bd3bb255e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0bf97fa640 0 client.5007 ms_handle_reset on v2:172.21.15.92:6835/1444537065 2024-04-22T07:04:45.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c233ae740 1 --2- 172.21.15.92:0/3760793519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bd3bb0e600 0x55bd3bba6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c233ae740 1 -- 172.21.15.92:0/3760793519 >> 172.21.15.92:0/3760793519 conn(0x55bd3bae8ec0 msgr2=0x55bd3bb0bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c233ae740 1 -- 172.21.15.92:0/3760793519 shutdown_connections 2024-04-22T07:04:45.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c233ae740 1 -- 172.21.15.92:0/3760793519 wait complete. 2024-04-22T07:04:45.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c233ae740 20 client.5007 trim_cache size 0 max 16384 2024-04-22T07:04:45.921 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:04:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "72eb29c5-e441-491c-9309-4fb4ef16ce27", 2024-04-22T07:04:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2024-04-22T07:04:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4269, 2024-04-22T07:04:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-04-22T07:04:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-04-22T07:04:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-04-22T07:04:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 32.008651100999998, 2024-04-22T07:04:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 20, 2024-04-22T07:04:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 20, 2024-04-22T07:04:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 49.014826249000002 2024-04-22T07:04:45.922 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:04:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c206e0640 1 -- 172.21.15.92:0/2590378022 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0bfc06c8f0 msgr2=0x7f0bfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c206e0640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0bfc06c8f0 0x7f0bfc06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c08036000 tx=0x7f0c0803b040 comp rx=0 tx=0).stop 2024-04-22T07:04:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c206e0640 1 -- 172.21.15.92:0/2590378022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c180c87e0 msgr2=0x7f0c18154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c206e0640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c180c87e0 0x7f0c18154fa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400cb40 tx=0x7f0c1400a060 comp rx=0 tx=0).stop 2024-04-22T07:04:45.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c206e0640 1 -- 172.21.15.92:0/2590378022 shutdown_connections 2024-04-22T07:04:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c206e0640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0bfc06c8f0 0x7f0bfc06edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c206e0640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c18158de0 0x7f0c1815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c206e0640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c18152d80 0x7f0c181554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c206e0640 1 --2- 172.21.15.92:0/2590378022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c180c87e0 0x7f0c18154fa0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c206e0640 1 -- 172.21.15.92:0/2590378022 >> 172.21.15.92:0/2590378022 conn(0x7f0c180cc220 msgr2=0x7f0c18151130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.916+0000 7f0c206e0640 1 -- 172.21.15.92:0/2590378022 shutdown_connections 2024-04-22T07:04:45.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:45.917+0000 7f0c206e0640 1 -- 172.21.15.92:0/2590378022 wait complete. 2024-04-22T07:04:45.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-22T07:04:46.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.024+0000 7f1820f8f640 1 Processor -- start 2024-04-22T07:04:46.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.024+0000 7f1820f8f640 1 -- start start 2024-04-22T07:04:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.025+0000 7f1820f8f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f181c067920 0x7f181c15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.025+0000 7f1820f8f640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f181c0c76d0 0x7f181c154b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.025+0000 7f1820f8f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f181c067d00 0x7f181c155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.025+0000 7f1820f8f640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f181c06d370 con 0x7f181c067920 2024-04-22T07:04:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.025+0000 7f1820f8f640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f181c0c7e40 con 0x7f181c0c76d0 2024-04-22T07:04:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.025+0000 7f1820f8f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f181c0c8100 con 0x7f181c067d00 2024-04-22T07:04:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.026+0000 7f181a59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f181c067920 0x7f181c15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.026+0000 7f181a59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f181c067920 0x7f181c15e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51808/0 (socket says 172.21.15.92:51808) 2024-04-22T07:04:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.026+0000 7f181a59c640 1 -- 172.21.15.92:0/1739164170 learned_addr learned my addr 172.21.15.92:0/1739164170 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:46.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.026+0000 7f181ad9d640 1 --2- 172.21.15.92:0/1739164170 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f181c067d00 0x7f181c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.026+0000 7f1819d9b640 1 --2- 172.21.15.92:0/1739164170 >> v2:172.21.15.145:3300/0 conn(0x7f181c0c76d0 0x7f181c154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.026+0000 7f181a59c640 1 -- 172.21.15.92:0/1739164170 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f181c067d00 msgr2=0x7f181c155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.026+0000 7f181a59c640 1 --2- 172.21.15.92:0/1739164170 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f181c067d00 0x7f181c155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.026+0000 7f181a59c640 1 -- 172.21.15.92:0/1739164170 >> v2:172.21.15.145:3300/0 conn(0x7f181c0c76d0 msgr2=0x7f181c154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.026+0000 7f181a59c640 1 --2- 172.21.15.92:0/1739164170 >> v2:172.21.15.145:3300/0 conn(0x7f181c0c76d0 0x7f181c154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.026+0000 7f181a59c640 1 -- 172.21.15.92:0/1739164170 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c06c890 con 0x7f181c067920 2024-04-22T07:04:46.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.026+0000 7f181ad9d640 1 --2- 172.21.15.92:0/1739164170 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f181c067d00 0x7f181c155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:46.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.026+0000 7f1819d9b640 1 --2- 172.21.15.92:0/1739164170 >> v2:172.21.15.145:3300/0 conn(0x7f181c0c76d0 0x7f181c154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f181a59c640 1 --2- 172.21.15.92:0/1739164170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f181c067920 0x7f181c15e2a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f181c15be40 tx=0x7f181402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8945040a7f06e2e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f181959a640 1 -- 172.21.15.92:0/1739164170 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181402cb00 con 0x7f181c067920 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f181959a640 1 -- 172.21.15.92:0/1739164170 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f181402cca0 con 0x7f181c067920 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f181959a640 1 -- 172.21.15.92:0/1739164170 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814034610 con 0x7f181c067920 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f1820f8f640 1 -- 172.21.15.92:0/1739164170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f181c067920 msgr2=0x7f181c15e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f1820f8f640 1 --2- 172.21.15.92:0/1739164170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f181c067920 0x7f181c15e2a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f181c15be40 tx=0x7f181402bf60 comp rx=0 tx=0).stop 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f1820f8f640 1 -- 172.21.15.92:0/1739164170 shutdown_connections 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f1820f8f640 1 --2- 172.21.15.92:0/1739164170 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f181c067d00 0x7f181c155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f1820f8f640 1 --2- 172.21.15.92:0/1739164170 >> v2:172.21.15.145:3300/0 conn(0x7f181c0c76d0 0x7f181c154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f1820f8f640 1 --2- 172.21.15.92:0/1739164170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f181c067920 0x7f181c15e2a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f1820f8f640 1 -- 172.21.15.92:0/1739164170 >> 172.21.15.92:0/1739164170 conn(0x7f181c0c9f40 msgr2=0x7f181c0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f1820f8f640 1 -- 172.21.15.92:0/1739164170 shutdown_connections 2024-04-22T07:04:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.027+0000 7f1820f8f640 1 -- 172.21.15.92:0/1739164170 wait complete. 2024-04-22T07:04:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.029+0000 7f1820f8f640 1 Processor -- start 2024-04-22T07:04:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.029+0000 7f1820f8f640 1 -- start start 2024-04-22T07:04:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.029+0000 7f1820f8f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f181c067920 0x7f181c0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.029+0000 7f1820f8f640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f181c067d00 0x7f181c157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.029+0000 7f1820f8f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f181c0c76d0 0x7f181c157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.029+0000 7f1820f8f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f181c06b970 con 0x7f181c0c76d0 2024-04-22T07:04:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.029+0000 7f1820f8f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f181c06ce50 con 0x7f181c067920 2024-04-22T07:04:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.029+0000 7f1820f8f640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f181c06baf0 con 0x7f181c067d00 2024-04-22T07:04:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.029+0000 7f181ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f181c0c76d0 0x7f181c157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.029+0000 7f181ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f181c0c76d0 0x7f181c157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51816/0 (socket says 172.21.15.92:51816) 2024-04-22T07:04:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.029+0000 7f181ad9d640 1 -- 172.21.15.92:0/1098860720 learned_addr learned my addr 172.21.15.92:0/1098860720 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f1819d9b640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f181c067d00 0x7f181c157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f181a59c640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f181c067920 0x7f181c0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f181ad9d640 1 -- 172.21.15.92:0/1098860720 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f181c067d00 msgr2=0x7f181c157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f181ad9d640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f181c067d00 0x7f181c157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f181ad9d640 1 -- 172.21.15.92:0/1098860720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f181c067920 msgr2=0x7f181c0c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f181ad9d640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f181c067920 0x7f181c0c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f181ad9d640 1 -- 172.21.15.92:0/1098860720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c1f15c0 con 0x7f181c0c76d0 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f1819d9b640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f181c067d00 0x7f181c157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f181a59c640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f181c067920 0x7f181c0c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f181ad9d640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f181c0c76d0 0x7f181c157550 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f180c00c5c0 tx=0x7f180c00ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f1812ffd640 1 -- 172.21.15.92:0/1098860720 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c015070 con 0x7f181c0c76d0 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f1812ffd640 1 -- 172.21.15.92:0/1098860720 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180c004240 con 0x7f181c0c76d0 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f1812ffd640 1 -- 172.21.15.92:0/1098860720 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c003610 con 0x7f181c0c76d0 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f1820f8f640 1 -- 172.21.15.92:0/1098860720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181c156170 con 0x7f181c0c76d0 2024-04-22T07:04:46.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.030+0000 7f1820f8f640 1 -- 172.21.15.92:0/1098860720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181c1f1250 con 0x7f181c0c76d0 2024-04-22T07:04:46.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.031+0000 7f1812ffd640 1 -- 172.21.15.92:0/1098860720 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f180c0037b0 con 0x7f181c0c76d0 2024-04-22T07:04:46.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.032+0000 7f1812ffd640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f17fc06c8f0 0x7f17fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.032+0000 7f1812ffd640 1 -- 172.21.15.92:0/1098860720 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f180c08b980 con 0x7f181c0c76d0 2024-04-22T07:04:46.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.032+0000 7f181a59c640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f17fc06c8f0 0x7f17fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.032+0000 7f181a59c640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f17fc06c8f0 0x7f17fc06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1814035000 tx=0x7f1814030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.033+0000 7f1823b7f740 1 Processor -- start 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.033+0000 7f1823b7f740 1 -- start start 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.033+0000 7f1823b7f740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c73951e40 0x559c7395af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1823b7f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73912ec0 0x559c7395b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1823b7f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c7390eca0 0x559c73936ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1823b7f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559c73875b10 con 0x559c73912ec0 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1823b7f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559c73813240 con 0x559c7390eca0 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1823b7f740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x559c73871ad0 con 0x559c73951e40 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1819d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73912ec0 0x559c7395b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1819d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73912ec0 0x559c7395b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51818/0 (socket says 172.21.15.92:51818) 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1819d9b640 1 -- 172.21.15.92:0/3620555475 learned_addr learned my addr 172.21.15.92:0/3620555475 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1819d9b640 1 -- 172.21.15.92:0/3620555475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c73951e40 msgr2=0x559c7395af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1819d9b640 1 --2- 172.21.15.92:0/3620555475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c73951e40 0x559c7395af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f181ad9d640 1 --2- 172.21.15.92:0/3620555475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c7390eca0 0x559c73936ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1819d9b640 1 -- 172.21.15.92:0/3620555475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c7390eca0 msgr2=0x559c73936ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1819d9b640 1 --2- 172.21.15.92:0/3620555475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c7390eca0 0x559c73936ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1819d9b640 1 -- 172.21.15.92:0/3620555475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c735882d0 con 0x559c73912ec0 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f1819d9b640 1 --2- 172.21.15.92:0/3620555475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73912ec0 0x559c7395b4a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f180400b940 tx=0x7f180400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def2401368d3f550 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f17fbfff640 1 -- 172.21.15.92:0/3620555475 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804016070 con 0x559c73912ec0 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f17fbfff640 1 -- 172.21.15.92:0/3620555475 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180400dc70 con 0x559c73912ec0 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.034+0000 7f17fbfff640 1 -- 172.21.15.92:0/3620555475 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804012680 con 0x559c73912ec0 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.035+0000 7f1823b7f740 1 -- 172.21.15.92:0/3620555475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73912ec0 msgr2=0x559c7395b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.035+0000 7f1823b7f740 1 --2- 172.21.15.92:0/3620555475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73912ec0 0x559c7395b4a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f180400b940 tx=0x7f180400be00 comp rx=0 tx=0).stop 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.035+0000 7f1823b7f740 1 -- 172.21.15.92:0/3620555475 shutdown_connections 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.035+0000 7f1823b7f740 1 --2- 172.21.15.92:0/3620555475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c7390eca0 0x559c73936ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.035+0000 7f1823b7f740 1 --2- 172.21.15.92:0/3620555475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73912ec0 0x559c7395b4a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.035+0000 7f1823b7f740 1 --2- 172.21.15.92:0/3620555475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c73951e40 0x559c7395af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.035+0000 7f1823b7f740 1 -- 172.21.15.92:0/3620555475 >> 172.21.15.92:0/3620555475 conn(0x559c7390d990 msgr2=0x559c73910290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.035+0000 7f1823b7f740 1 -- 172.21.15.92:0/3620555475 shutdown_connections 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.035+0000 7f1823b7f740 1 -- 172.21.15.92:0/3620555475 wait complete. 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.035+0000 7f1823b7f740 1 -- start start 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.036+0000 7f1823b7f740 1 Processor -- start 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.036+0000 7f1823b7f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c73937680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.036+0000 7f1823b7f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 0x559c739635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.036+0000 7f1823b7f740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c7390c530 0x559c73957160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.036+0000 7f1823b7f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559c73911250 con 0x559c73951e40 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.036+0000 7f1823b7f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559c737fe190 con 0x559c73912ec0 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.036+0000 7f1823b7f740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x559c73911530 con 0x559c7390c530 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.036+0000 7f1823b7f740 10 client.0 _ll_register_callbacks cb 0x559c737f98b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.036+0000 7f1819d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 0x559c739635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f1819d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 0x559c739635f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51824/0 (socket says 172.21.15.92:51824) 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f1819d9b640 1 -- 172.21.15.92:0/4178038106 learned_addr learned my addr 172.21.15.92:0/4178038106 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f181a59c640 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c73937680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f1819d9b640 1 -- 172.21.15.92:0/4178038106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c7390c530 msgr2=0x559c73957160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f181ad9d640 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c7390c530 0x559c73957160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f1819d9b640 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c7390c530 0x559c73957160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f1819d9b640 1 -- 172.21.15.92:0/4178038106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 msgr2=0x559c73937680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f1819d9b640 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c73937680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f1819d9b640 1 -- 172.21.15.92:0/4178038106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c736526b0 con 0x559c73951e40 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f1819d9b640 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 0x559c739635f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f180400b190 tx=0x7f1804004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bcb87fdcf8e500 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f17f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f17f97fa640 1 -- 172.21.15.92:0/4178038106 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804016040 con 0x559c73951e40 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f17f97fa640 1 -- 172.21.15.92:0/4178038106 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1804012cc0 con 0x559c73951e40 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f1823b7f740 1 -- 172.21.15.92:0/4178038106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f180400dc70 con 0x559c73951e40 2024-04-22T07:04:46.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f17f97fa640 1 -- 172.21.15.92:0/4178038106 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f180401b450 con 0x559c73951e40 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f1823b7f740 10 client.5019 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f1823b7f740 1 -- 172.21.15.92:0/4178038106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559c73958910 con 0x559c73951e40 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f17f97fa640 1 -- 172.21.15.92:0/4178038106 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180400dc70 con 0x559c73951e40 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.037+0000 7f17f97fa640 1 -- 172.21.15.92:0/4178038106 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f180401b880 con 0x559c73951e40 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.038+0000 7f1823b7f740 10 client.5019 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.038+0000 7f1823b7f740 10 client.5019 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.038+0000 7f1823b7f740 20 client.5019 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.038+0000 7f1823b7f740 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x559c7396fa40 0x559c73971e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.038+0000 7f1823b7f740 4 client.5019 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.038+0000 7f1823b7f740 1 -- 172.21.15.92:0/4178038106 --> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c73883530 con 0x559c7396fa40 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.038+0000 7f181a59c640 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x559c7396fa40 0x559c73971e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.038+0000 7f17f97fa640 10 client.5019 ms_handle_connect on v2:172.21.15.92:6834/101798872 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.038+0000 7f181a59c640 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x559c7396fa40 0x559c73971e30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7f494ebf4eb2dd0 server_cookie=a158b12f865138ec in_seq=0 out_seq=0 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f17f97fa640 1 -- 172.21.15.92:0/4178038106 <== mds.1 v2:172.21.15.92:6834/101798872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c73883530 con 0x559c7396fa40 2024-04-22T07:04:46.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f17f97fa640 10 client.5019 handle_command_reply: tid=0 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f1823b7f740 1 client.5019 shutdown 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f1823b7f740 1 -- 172.21.15.92:0/4178038106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 msgr2=0x559c739635f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f1823b7f740 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 0x559c739635f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f180400b190 tx=0x7f1804004730 comp rx=0 tx=0).stop 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f1823b7f740 1 -- 172.21.15.92:0/4178038106 shutdown_connections 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f1823b7f740 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x559c7396fa40 0x559c73971e30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f1823b7f740 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c7390c530 0x559c73957160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f1823b7f740 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 0x559c739635f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f17f97fa640 0 client.5019 ms_handle_reset on v2:172.21.15.92:6834/101798872 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f1823b7f740 1 --2- 172.21.15.92:0/4178038106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c73937680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f1823b7f740 1 -- 172.21.15.92:0/4178038106 >> 172.21.15.92:0/4178038106 conn(0x559c7390eca0 msgr2=0x559c73951440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f1823b7f740 1 -- 172.21.15.92:0/4178038106 shutdown_connections 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f1823b7f740 1 -- 172.21.15.92:0/4178038106 wait complete. 2024-04-22T07:04:46.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.039+0000 7f1823b7f740 20 client.5019 trim_cache size 0 max 16384 2024-04-22T07:04:46.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.057+0000 7f1823b7f740 1 Processor -- start 2024-04-22T07:04:46.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.057+0000 7f1823b7f740 1 -- start start 2024-04-22T07:04:46.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f1823b7f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c7390c530 0x559c73935ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f1823b7f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c739363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f1823b7f740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c73951e40 0x559c73949ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f1823b7f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1fca40 con 0x559c7390c530 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f1823b7f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1fd110 con 0x559c73912ec0 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f1823b7f740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f181c143790 con 0x559c73951e40 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f181a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c7390c530 0x559c73935ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f181a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c7390c530 0x559c73935ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51830/0 (socket says 172.21.15.92:51830) 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f181a59c640 1 -- 172.21.15.92:0/439880029 learned_addr learned my addr 172.21.15.92:0/439880029 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f181ad9d640 1 --2- 172.21.15.92:0/439880029 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c73951e40 0x559c73949ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f1819d9b640 1 --2- 172.21.15.92:0/439880029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c739363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f181a59c640 1 -- 172.21.15.92:0/439880029 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c73951e40 msgr2=0x559c73949ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f181a59c640 1 --2- 172.21.15.92:0/439880029 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c73951e40 0x559c73949ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f181a59c640 1 -- 172.21.15.92:0/439880029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 msgr2=0x559c739363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f181a59c640 1 --2- 172.21.15.92:0/439880029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c739363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f181a59c640 1 -- 172.21.15.92:0/439880029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c7395ab00 con 0x559c7390c530 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f1819d9b640 1 --2- 172.21.15.92:0/439880029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c739363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f181a59c640 1 --2- 172.21.15.92:0/439880029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c7390c530 0x559c73935ea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f181402c570 tx=0x7f181402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4357cc16e4d1cdad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f17fb7fe640 1 -- 172.21.15.92:0/439880029 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814040be0 con 0x559c7390c530 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f17fb7fe640 1 -- 172.21.15.92:0/439880029 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1814054070 con 0x559c7390c530 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.058+0000 7f17fb7fe640 1 -- 172.21.15.92:0/439880029 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181404e050 con 0x559c7390c530 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.059+0000 7f1823b7f740 1 -- 172.21.15.92:0/439880029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c7390c530 msgr2=0x559c73935ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.059+0000 7f1823b7f740 1 --2- 172.21.15.92:0/439880029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c7390c530 0x559c73935ea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f181402c570 tx=0x7f181402e000 comp rx=0 tx=0).stop 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.059+0000 7f1823b7f740 1 -- 172.21.15.92:0/439880029 shutdown_connections 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.059+0000 7f1823b7f740 1 --2- 172.21.15.92:0/439880029 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c73951e40 0x559c73949ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.059+0000 7f1823b7f740 1 --2- 172.21.15.92:0/439880029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c739363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.059+0000 7f1823b7f740 1 --2- 172.21.15.92:0/439880029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c7390c530 0x559c73935ea0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.059+0000 7f1823b7f740 1 -- 172.21.15.92:0/439880029 >> 172.21.15.92:0/439880029 conn(0x559c7390eca0 msgr2=0x559c73958ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.059+0000 7f1823b7f740 1 -- 172.21.15.92:0/439880029 shutdown_connections 2024-04-22T07:04:46.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.059+0000 7f1823b7f740 1 -- 172.21.15.92:0/439880029 wait complete. 2024-04-22T07:04:46.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.059+0000 7f1823b7f740 1 -- start start 2024-04-22T07:04:46.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.060+0000 7f1823b7f740 1 Processor -- start 2024-04-22T07:04:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.060+0000 7f1823b7f740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c7390c530 0x559c739624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.060+0000 7f1823b7f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c739d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.060+0000 7f1823b7f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 0x559c739c6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.060+0000 7f1823b7f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559c739584a0 con 0x559c73951e40 2024-04-22T07:04:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.060+0000 7f1823b7f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559c739576f0 con 0x559c73912ec0 2024-04-22T07:04:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.060+0000 7f1823b7f740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x559c7394b6f0 con 0x559c7390c530 2024-04-22T07:04:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f1823b7f740 10 client.0 _ll_register_callbacks cb 0x7f180400b7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f181ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 0x559c739c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f181ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 0x559c739c6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51832/0 (socket says 172.21.15.92:51832) 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f181ad9d640 1 -- 172.21.15.92:0/2709434913 learned_addr learned my addr 172.21.15.92:0/2709434913 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f181a59c640 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c7390c530 0x559c739624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f1819d9b640 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c739d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f181ad9d640 1 -- 172.21.15.92:0/2709434913 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c7390c530 msgr2=0x559c739624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f181ad9d640 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c7390c530 0x559c739624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f181ad9d640 1 -- 172.21.15.92:0/2709434913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 msgr2=0x559c739d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f181ad9d640 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c739d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f181ad9d640 1 -- 172.21.15.92:0/2709434913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c739121b0 con 0x559c73951e40 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f181ad9d640 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 0x559c739c6250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f180c0595b0 tx=0x7f180c059e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d66e0af84e84e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f17f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f17f97fa640 1 -- 172.21.15.92:0/2709434913 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c01e040 con 0x559c73951e40 2024-04-22T07:04:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f17f97fa640 1 -- 172.21.15.92:0/2709434913 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f180c058a70 con 0x559c73951e40 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f1823b7f740 1 -- 172.21.15.92:0/2709434913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1814054070 con 0x559c73951e40 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f17f97fa640 1 -- 172.21.15.92:0/2709434913 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f180c078050 con 0x559c73951e40 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f1823b7f740 10 client.5025 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f1823b7f740 1 -- 172.21.15.92:0/2709434913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559c739cfcd0 con 0x559c73951e40 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.061+0000 7f17f97fa640 1 -- 172.21.15.92:0/2709434913 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1814054070 con 0x559c73951e40 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f17f97fa640 1 -- 172.21.15.92:0/2709434913 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f180c055370 con 0x559c73951e40 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f1823b7f740 10 client.5025 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f1823b7f740 10 client.5025 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f1823b7f740 20 client.5025 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f1823b7f740 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x559c7394ecd0 0x559c739d8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f1823b7f740 4 client.5025 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f1823b7f740 1 -- 172.21.15.92:0/2709434913 --> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] -- command(tid 0: {"prefix": "status"}) -- 0x559c7387d3a0 con 0x559c7394ecd0 2024-04-22T07:04:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f181a59c640 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x559c7394ecd0 0x559c739d8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f17f97fa640 10 client.5025 ms_handle_connect on v2:172.21.15.92:6834/101798872 2024-04-22T07:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f181a59c640 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x559c7394ecd0 0x559c739d8ed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61eb5a41bd39481e server_cookie=d0f75a367aad5e17 in_seq=0 out_seq=0 2024-04-22T07:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f17f97fa640 1 -- 172.21.15.92:0/2709434913 <== mds.1 v2:172.21.15.92:6834/101798872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x559c7387d3a0 con 0x559c7394ecd0 2024-04-22T07:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f17f97fa640 10 client.5025 handle_command_reply: tid=0 2024-04-22T07:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f1823b7f740 1 client.5025 shutdown 2024-04-22T07:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f1823b7f740 1 -- 172.21.15.92:0/2709434913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 msgr2=0x559c739c6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f1823b7f740 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 0x559c739c6250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f180c0595b0 tx=0x7f180c059e90 comp rx=0 tx=0).stop 2024-04-22T07:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.062+0000 7f1823b7f740 1 -- 172.21.15.92:0/2709434913 shutdown_connections 2024-04-22T07:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1823b7f740 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.92:6834/101798872,v1:172.21.15.92:6837/101798872] conn(0x559c7394ecd0 0x559c739d8ed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1823b7f740 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c73951e40 0x559c739c6250 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1823b7f740 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559c73912ec0 0x559c739d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f17f97fa640 0 client.5025 ms_handle_reset on v2:172.21.15.92:6834/101798872 2024-04-22T07:04:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1823b7f740 1 --2- 172.21.15.92:0/2709434913 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559c7390c530 0x559c739624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1823b7f740 1 -- 172.21.15.92:0/2709434913 >> 172.21.15.92:0/2709434913 conn(0x559c7390eca0 msgr2=0x559c73936ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1823b7f740 1 -- 172.21.15.92:0/2709434913 shutdown_connections 2024-04-22T07:04:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1823b7f740 1 -- 172.21.15.92:0/2709434913 wait complete. 2024-04-22T07:04:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1823b7f740 20 client.5025 trim_cache size 0 max 16384 2024-04-22T07:04:46.068 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:04:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "72eb29c5-e441-491c-9309-4fb4ef16ce27", 2024-04-22T07:04:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2024-04-22T07:04:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4273, 2024-04-22T07:04:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-04-22T07:04:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-04-22T07:04:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-04-22T07:04:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 37.250808923999998, 2024-04-22T07:04:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 20, 2024-04-22T07:04:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 20, 2024-04-22T07:04:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 49.161524290999999 2024-04-22T07:04:46.069 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:04:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1820f8f640 1 -- 172.21.15.92:0/1098860720 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f17fc06c8f0 msgr2=0x7f17fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1820f8f640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f17fc06c8f0 0x7f17fc06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1814035000 tx=0x7f1814030000 comp rx=0 tx=0).stop 2024-04-22T07:04:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1820f8f640 1 -- 172.21.15.92:0/1098860720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f181c0c76d0 msgr2=0x7f181c157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1820f8f640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f181c0c76d0 0x7f181c157550 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f180c00c5c0 tx=0x7f180c00ca80 comp rx=0 tx=0).stop 2024-04-22T07:04:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1820f8f640 1 -- 172.21.15.92:0/1098860720 shutdown_connections 2024-04-22T07:04:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1820f8f640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f17fc06c8f0 0x7f17fc06edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1820f8f640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f181c0c76d0 0x7f181c157550 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1820f8f640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f181c067d00 0x7f181c157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1820f8f640 1 --2- 172.21.15.92:0/1098860720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f181c067920 0x7f181c0c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1820f8f640 1 -- 172.21.15.92:0/1098860720 >> 172.21.15.92:0/1098860720 conn(0x7f181c0c9f40 msgr2=0x7f181c153870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1820f8f640 1 -- 172.21.15.92:0/1098860720 shutdown_connections 2024-04-22T07:04:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.063+0000 7f1820f8f640 1 -- 172.21.15.92:0/1098860720 wait complete. 2024-04-22T07:04:46.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-22T07:04:46.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.172+0000 7fc1559e2640 1 Processor -- start 2024-04-22T07:04:46.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.172+0000 7fc1559e2640 1 -- start start 2024-04-22T07:04:46.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.172+0000 7fc1559e2640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc150154fa0 0x7fc15015b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.172+0000 7fc1559e2640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1500cb8c0 0x7fc1500b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.172+0000 7fc1559e2640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc15015e4a0 con 0x7fc1500cb480 2024-04-22T07:04:46.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.172+0000 7fc1559e2640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fc15015e780 con 0x7fc1500cb8c0 2024-04-22T07:04:46.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.172+0000 7fc1559e2640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc15015ea60 con 0x7fc150154fa0 2024-04-22T07:04:46.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.173+0000 7fc14f7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1500cb8c0 0x7fc1500b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.173+0000 7fc14f7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1500cb8c0 0x7fc1500b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51844/0 (socket says 172.21.15.92:51844) 2024-04-22T07:04:46.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.173+0000 7fc14f7fe640 1 -- 172.21.15.92:0/2717882596 learned_addr learned my addr 172.21.15.92:0/2717882596 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:46.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.173+0000 7fc14effd640 1 --2- 172.21.15.92:0/2717882596 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc150154fa0 0x7fc15015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.173+0000 7fc14f7fe640 1 -- 172.21.15.92:0/2717882596 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc150154fa0 msgr2=0x7fc15015b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.173+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/2717882596 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc150154fa0 0x7fc15015b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.173+0000 7fc14f7fe640 1 -- 172.21.15.92:0/2717882596 >> v1:172.21.15.145:6789/0 conn(0x7fc1500cb480 legacy=0x7fc1500d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.173+0000 7fc14f7fe640 1 -- 172.21.15.92:0/2717882596 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15015ed40 con 0x7fc1500cb8c0 2024-04-22T07:04:46.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.174+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/2717882596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1500cb8c0 0x7fc1500b4970 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc148009e50 tx=0x7fc14802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b18e93c148dbe70 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.174+0000 7fc14e7fc640 1 -- 172.21.15.92:0/2717882596 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14802cb90 con 0x7fc1500cb8c0 2024-04-22T07:04:46.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.174+0000 7fc14e7fc640 1 -- 172.21.15.92:0/2717882596 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc14802cd30 con 0x7fc1500cb8c0 2024-04-22T07:04:46.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.174+0000 7fc14e7fc640 1 -- 172.21.15.92:0/2717882596 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148034610 con 0x7fc1500cb8c0 2024-04-22T07:04:46.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.174+0000 7fc1559e2640 1 -- 172.21.15.92:0/2717882596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1500cb8c0 msgr2=0x7fc1500b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.174+0000 7fc1559e2640 1 --2- 172.21.15.92:0/2717882596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1500cb8c0 0x7fc1500b4970 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc148009e50 tx=0x7fc14802bff0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.175+0000 7fc1559e2640 1 -- 172.21.15.92:0/2717882596 shutdown_connections 2024-04-22T07:04:46.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.175+0000 7fc1559e2640 1 --2- 172.21.15.92:0/2717882596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1500cb8c0 0x7fc1500b4970 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.175+0000 7fc1559e2640 1 --2- 172.21.15.92:0/2717882596 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc150154fa0 0x7fc15015b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.175+0000 7fc1559e2640 1 -- 172.21.15.92:0/2717882596 >> 172.21.15.92:0/2717882596 conn(0x7fc150073930 msgr2=0x7fc15006cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.175+0000 7fc1559e2640 1 -- 172.21.15.92:0/2717882596 shutdown_connections 2024-04-22T07:04:46.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.175+0000 7fc1559e2640 1 -- 172.21.15.92:0/2717882596 wait complete. 2024-04-22T07:04:46.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.176+0000 7fc1559e2640 1 Processor -- start 2024-04-22T07:04:46.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.176+0000 7fc1559e2640 1 -- start start 2024-04-22T07:04:46.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.176+0000 7fc1559e2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1500cb480 0x7fc1500d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.176+0000 7fc1559e2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1500cb8c0 0x7fc150157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.176+0000 7fc1559e2640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc150154fa0 0x7fc15014ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.176+0000 7fc1559e2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc150155380 con 0x7fc1500cb8c0 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.176+0000 7fc1559e2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc1500d0ba0 con 0x7fc1500cb480 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.176+0000 7fc1559e2640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc1501577d0 con 0x7fc150154fa0 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc1467fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1500cb8c0 0x7fc150157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc1467fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1500cb8c0 0x7fc150157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51850/0 (socket says 172.21.15.92:51850) 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc1467fc640 1 -- 172.21.15.92:0/2421594228 learned_addr learned my addr 172.21.15.92:0/2421594228 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc150154fa0 0x7fc15014ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc14effd640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1500cb480 0x7fc1500d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc1467fc640 1 -- 172.21.15.92:0/2421594228 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc150154fa0 msgr2=0x7fc15014ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc1467fc640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc150154fa0 0x7fc15014ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc1467fc640 1 -- 172.21.15.92:0/2421594228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1500cb480 msgr2=0x7fc1500d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc1467fc640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1500cb480 0x7fc1500d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc1467fc640 1 -- 172.21.15.92:0/2421594228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1501f13b0 con 0x7fc1500cb8c0 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc150154fa0 0x7fc15014ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc14effd640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1500cb480 0x7fc1500d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:46.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc1467fc640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1500cb8c0 0x7fc150157f40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc13800c3f0 tx=0x7fc13800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc14cff9640 1 -- 172.21.15.92:0/2421594228 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380052d0 con 0x7fc1500cb8c0 2024-04-22T07:04:46.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc14cff9640 1 -- 172.21.15.92:0/2421594228 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc138005490 con 0x7fc1500cb8c0 2024-04-22T07:04:46.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc1559e2640 1 -- 172.21.15.92:0/2421594228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc150155810 con 0x7fc1500cb8c0 2024-04-22T07:04:46.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc14cff9640 1 -- 172.21.15.92:0/2421594228 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138007260 con 0x7fc1500cb8c0 2024-04-22T07:04:46.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.177+0000 7fc1559e2640 1 -- 172.21.15.92:0/2421594228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1501f1650 con 0x7fc1500cb8c0 2024-04-22T07:04:46.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.179+0000 7fc14cff9640 1 -- 172.21.15.92:0/2421594228 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc138007400 con 0x7fc1500cb8c0 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.179+0000 7fc14cff9640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc13006c920 0x7fc13006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.179+0000 7fc14cff9640 1 -- 172.21.15.92:0/2421594228 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc13808b010 con 0x7fc1500cb8c0 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.179+0000 7fc14effd640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc13006c920 0x7fc13006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.180+0000 7fc14effd640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc13006c920 0x7fc13006ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc140006fd0 tx=0x7fc140008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.180+0000 7fc1585cb740 1 Processor -- start 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.180+0000 7fc1585cb740 1 -- start start 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1585cb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a125f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1585cb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a0ddec0 0x55c01a1264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1585cb740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0d9ca0 0x55c01a101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1585cb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c01a040b10 con 0x55c01a0ddec0 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1585cb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c019fde240 con 0x55c01a11ce40 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1585cb740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55c01a03cad0 con 0x55c01a0d9ca0 2024-04-22T07:04:46.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1467fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a0ddec0 0x55c01a1264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1467fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a0ddec0 0x55c01a1264a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51858/0 (socket says 172.21.15.92:51858) 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1467fc640 1 -- 172.21.15.92:0/4123415035 learned_addr learned my addr 172.21.15.92:0/4123415035 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc14effd640 1 --2- 172.21.15.92:0/4123415035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a125f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/4123415035 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0d9ca0 0x55c01a101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1467fc640 1 -- 172.21.15.92:0/4123415035 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0d9ca0 msgr2=0x55c01a101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1467fc640 1 --2- 172.21.15.92:0/4123415035 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0d9ca0 0x55c01a101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1467fc640 1 -- 172.21.15.92:0/4123415035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 msgr2=0x55c01a125f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1467fc640 1 --2- 172.21.15.92:0/4123415035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a125f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1467fc640 1 -- 172.21.15.92:0/4123415035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c019d532d0 con 0x55c01a0ddec0 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.181+0000 7fc1467fc640 1 --2- 172.21.15.92:0/4123415035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a0ddec0 0x55c01a1264a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc13800c3c0 tx=0x7fc13800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8849f967c00465c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.182+0000 7fc1457fa640 1 -- 172.21.15.92:0/4123415035 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138057040 con 0x55c01a0ddec0 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.182+0000 7fc1457fa640 1 -- 172.21.15.92:0/4123415035 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1380553b0 con 0x55c01a0ddec0 2024-04-22T07:04:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.182+0000 7fc1457fa640 1 -- 172.21.15.92:0/4123415035 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138071320 con 0x55c01a0ddec0 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.182+0000 7fc1585cb740 1 -- 172.21.15.92:0/4123415035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a0ddec0 msgr2=0x55c01a1264a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.182+0000 7fc1585cb740 1 --2- 172.21.15.92:0/4123415035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a0ddec0 0x55c01a1264a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc13800c3c0 tx=0x7fc13800a000 comp rx=0 tx=0).stop 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.182+0000 7fc1585cb740 1 -- 172.21.15.92:0/4123415035 shutdown_connections 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.182+0000 7fc1585cb740 1 --2- 172.21.15.92:0/4123415035 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0d9ca0 0x55c01a101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.182+0000 7fc1585cb740 1 --2- 172.21.15.92:0/4123415035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a0ddec0 0x55c01a1264a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.182+0000 7fc1585cb740 1 --2- 172.21.15.92:0/4123415035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a125f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.182+0000 7fc1585cb740 1 -- 172.21.15.92:0/4123415035 >> 172.21.15.92:0/4123415035 conn(0x55c01a0d8990 msgr2=0x55c01a0db290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.182+0000 7fc1585cb740 1 -- 172.21.15.92:0/4123415035 shutdown_connections 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.182+0000 7fc1585cb740 1 -- 172.21.15.92:0/4123415035 wait complete. 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.183+0000 7fc1585cb740 1 -- start start 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.183+0000 7fc1585cb740 1 Processor -- start 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc1585cb740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 0x55c01a125740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc1585cb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a131110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc1585cb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a116710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc1585cb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c01a0dc250 con 0x55c01a124b60 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc1585cb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c019fc9190 con 0x55c01a11ce40 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc1585cb740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55c01a0dc530 con 0x55c01a0ddec0 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc1585cb740 10 client.0 _ll_register_callbacks cb 0x55c019fc48b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc14f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a116710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc14f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a116710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51868/0 (socket says 172.21.15.92:51868) 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc14f7fe640 1 -- 172.21.15.92:0/1714128429 learned_addr learned my addr 172.21.15.92:0/1714128429 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc14effd640 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 0x55c01a125740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc1467fc640 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a131110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc14f7fe640 1 -- 172.21.15.92:0/1714128429 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 msgr2=0x55c01a125740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 0x55c01a125740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc14f7fe640 1 -- 172.21.15.92:0/1714128429 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 msgr2=0x55c01a131110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a131110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc14f7fe640 1 -- 172.21.15.92:0/1714128429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c019e1d6b0 con 0x55c01a124b60 2024-04-22T07:04:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a116710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc14802c600 tx=0x7fc148004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e0c8e263c9c858 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc12e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc12e7fc640 1 -- 172.21.15.92:0/1714128429 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148009510 con 0x55c01a124b60 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc12e7fc640 1 -- 172.21.15.92:0/1714128429 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc148034b30 con 0x55c01a124b60 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc1585cb740 1 -- 172.21.15.92:0/1714128429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1380553b0 con 0x55c01a124b60 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc12e7fc640 1 -- 172.21.15.92:0/1714128429 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc148034d30 con 0x55c01a124b60 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc1585cb740 10 client.5037 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.184+0000 7fc1585cb740 1 -- 172.21.15.92:0/1714128429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55c01a124730 con 0x55c01a124b60 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.185+0000 7fc12e7fc640 1 -- 172.21.15.92:0/1714128429 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1380553b0 con 0x55c01a124b60 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.185+0000 7fc12e7fc640 1 -- 172.21.15.92:0/1714128429 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fc148052080 con 0x55c01a124b60 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.185+0000 7fc1585cb740 10 client.5037 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.185+0000 7fc1585cb740 10 client.5037 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.185+0000 7fc1585cb740 20 client.5037 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.185+0000 7fc1585cb740 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c01a0fc770 0x55c01a13c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.185+0000 7fc1585cb740 4 client.5037 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.185+0000 7fc1585cb740 1 -- 172.21.15.92:0/1714128429 --> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c01a04e530 con 0x55c01a0fc770 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.185+0000 7fc14effd640 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c01a0fc770 0x55c01a13c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.185+0000 7fc12e7fc640 10 client.5037 ms_handle_connect on v2:172.21.15.92:6838/3493022212 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.185+0000 7fc14effd640 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c01a0fc770 0x55c01a13c9b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17537aedf609ac60 server_cookie=ca149a4274e7fe7f in_seq=0 out_seq=0 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc12e7fc640 1 -- 172.21.15.92:0/1714128429 <== mds.0 v2:172.21.15.92:6838/3493022212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c01a04e530 con 0x55c01a0fc770 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc12e7fc640 10 client.5037 handle_command_reply: tid=0 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc1585cb740 1 client.5037 shutdown 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc1585cb740 1 -- 172.21.15.92:0/1714128429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 msgr2=0x55c01a116710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc1585cb740 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a116710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc14802c600 tx=0x7fc148004ab0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc1585cb740 1 -- 172.21.15.92:0/1714128429 shutdown_connections 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc1585cb740 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c01a0fc770 0x55c01a13c9b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc1585cb740 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a116710 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc1585cb740 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a131110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc12e7fc640 0 client.5037 ms_handle_reset on v2:172.21.15.92:6838/3493022212 2024-04-22T07:04:46.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc1585cb740 1 --2- 172.21.15.92:0/1714128429 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 0x55c01a125740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc1585cb740 1 -- 172.21.15.92:0/1714128429 >> 172.21.15.92:0/1714128429 conn(0x55c01a0d9ca0 msgr2=0x55c01a11c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc1585cb740 1 -- 172.21.15.92:0/1714128429 shutdown_connections 2024-04-22T07:04:46.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc1585cb740 1 -- 172.21.15.92:0/1714128429 wait complete. 2024-04-22T07:04:46.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.186+0000 7fc1585cb740 20 client.5037 trim_cache size 0 max 16384 2024-04-22T07:04:46.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.204+0000 7fc1585cb740 1 Processor -- start 2024-04-22T07:04:46.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.204+0000 7fc1585cb740 1 -- start start 2024-04-22T07:04:46.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc1585cb740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 0x55c01a125cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc1585cb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a126210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc1585cb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a116070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc1585cb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501f3e00 con 0x55c01a124b60 2024-04-22T07:04:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc1585cb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c01a0fb470 con 0x55c01a11ce40 2024-04-22T07:04:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc1585cb740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55c01a11a9d0 con 0x55c01a0ddec0 2024-04-22T07:04:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc14f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc14f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51880/0 (socket says 172.21.15.92:51880) 2024-04-22T07:04:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc14f7fe640 1 -- 172.21.15.92:0/643679518 learned_addr learned my addr 172.21.15.92:0/643679518 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc14f7fe640 1 -- 172.21.15.92:0/643679518 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 msgr2=0x55c01a125cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/643679518 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 0x55c01a125cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc14f7fe640 1 -- 172.21.15.92:0/643679518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 msgr2=0x55c01a126210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/643679518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a126210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc14f7fe640 1 -- 172.21.15.92:0/643679518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c01a0d9720 con 0x55c01a124b60 2024-04-22T07:04:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.205+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/643679518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a116070 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc1480093e0 tx=0x7fc14803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9630e9b071bb441f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.206+0000 7fc144ff9640 1 -- 172.21.15.92:0/643679518 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148009510 con 0x55c01a124b60 2024-04-22T07:04:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.206+0000 7fc144ff9640 1 -- 172.21.15.92:0/643679518 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc148009af0 con 0x55c01a124b60 2024-04-22T07:04:46.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.206+0000 7fc144ff9640 1 -- 172.21.15.92:0/643679518 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148034b50 con 0x55c01a124b60 2024-04-22T07:04:46.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.206+0000 7fc1585cb740 1 -- 172.21.15.92:0/643679518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 msgr2=0x55c01a116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.206+0000 7fc1585cb740 1 --2- 172.21.15.92:0/643679518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a116070 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc1480093e0 tx=0x7fc14803f000 comp rx=0 tx=0).stop 2024-04-22T07:04:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.206+0000 7fc1585cb740 1 -- 172.21.15.92:0/643679518 shutdown_connections 2024-04-22T07:04:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.206+0000 7fc1585cb740 1 --2- 172.21.15.92:0/643679518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a116070 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.206+0000 7fc1585cb740 1 --2- 172.21.15.92:0/643679518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a126210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.206+0000 7fc1585cb740 1 --2- 172.21.15.92:0/643679518 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 0x55c01a125cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.206+0000 7fc1585cb740 1 -- 172.21.15.92:0/643679518 >> 172.21.15.92:0/643679518 conn(0x55c01a0d9ca0 msgr2=0x55c01a11bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.206+0000 7fc1585cb740 1 -- 172.21.15.92:0/643679518 shutdown_connections 2024-04-22T07:04:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.206+0000 7fc1585cb740 1 -- 172.21.15.92:0/643679518 wait complete. 2024-04-22T07:04:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.207+0000 7fc1585cb740 1 -- start start 2024-04-22T07:04:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.207+0000 7fc1585cb740 1 Processor -- start 2024-04-22T07:04:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc1585cb740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 0x55c01a115060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc1585cb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc1585cb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a115e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc1585cb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c01a0fc5f0 con 0x55c01a124b60 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc1585cb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c01a1239e0 con 0x55c01a11ce40 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc1585cb740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55c01a0fcc60 con 0x55c01a0ddec0 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc1585cb740 10 client.0 _ll_register_callbacks cb 0x7fc148009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc14f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a115e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc14f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a115e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51890/0 (socket says 172.21.15.92:51890) 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc14f7fe640 1 -- 172.21.15.92:0/22031425 learned_addr learned my addr 172.21.15.92:0/22031425 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc1467fc640 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc14effd640 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 0x55c01a115060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc14f7fe640 1 -- 172.21.15.92:0/22031425 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 msgr2=0x55c01a115060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 0x55c01a115060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc14f7fe640 1 -- 172.21.15.92:0/22031425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 msgr2=0x55c01a101fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a101fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc14f7fe640 1 -- 172.21.15.92:0/22031425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c01a142590 con 0x55c01a124b60 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc1467fc640 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc14f7fe640 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a115e50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc148002230 tx=0x7fc148009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48549abe33a545b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc12e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc12e7fc640 1 -- 172.21.15.92:0/22031425 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148009510 con 0x55c01a124b60 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc1585cb740 1 -- 172.21.15.92:0/22031425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc148009af0 con 0x55c01a124b60 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc12e7fc640 1 -- 172.21.15.92:0/22031425 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc148063ad0 con 0x55c01a124b60 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc12e7fc640 1 -- 172.21.15.92:0/22031425 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc148063cf0 con 0x55c01a124b60 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc1585cb740 10 client.5043 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.208+0000 7fc1585cb740 1 -- 172.21.15.92:0/22031425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55c01a124310 con 0x55c01a124b60 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc12e7fc640 1 -- 172.21.15.92:0/22031425 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc148009af0 con 0x55c01a124b60 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc12e7fc640 1 -- 172.21.15.92:0/22031425 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fc148052040 con 0x55c01a124b60 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc1585cb740 10 client.5043 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:46.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc1585cb740 10 client.5043 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc1585cb740 20 client.5043 populate_metadata read hostname 'smithi092' 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc1585cb740 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c01a125cf0 0x55c01a1260d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc1585cb740 4 client.5043 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc1585cb740 1 -- 172.21.15.92:0/22031425 --> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] -- command(tid 0: {"prefix": "status"}) -- 0x55c01a0483a0 con 0x55c01a125cf0 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc14effd640 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c01a125cf0 0x55c01a1260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc14effd640 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c01a125cf0 0x55c01a1260d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7907337dad395c server_cookie=d4ecbea231ffad5f in_seq=0 out_seq=0 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc12e7fc640 10 client.5043 ms_handle_connect on v2:172.21.15.92:6838/3493022212 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc12e7fc640 1 -- 172.21.15.92:0/22031425 <== mds.0 v2:172.21.15.92:6838/3493022212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55c01a0483a0 con 0x55c01a125cf0 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc12e7fc640 10 client.5043 handle_command_reply: tid=0 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc1585cb740 1 client.5043 shutdown 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc1585cb740 1 -- 172.21.15.92:0/22031425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 msgr2=0x55c01a115e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.209+0000 7fc1585cb740 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a115e50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc148002230 tx=0x7fc148009000 comp rx=0 tx=0).stop 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1585cb740 1 -- 172.21.15.92:0/22031425 shutdown_connections 2024-04-22T07:04:46.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1585cb740 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c01a125cf0 0x55c01a1260d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1585cb740 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01a124b60 0x55c01a115e50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1585cb740 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c01a11ce40 0x55c01a101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc12e7fc640 0 client.5043 ms_handle_reset on v2:172.21.15.92:6838/3493022212 2024-04-22T07:04:46.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1585cb740 1 --2- 172.21.15.92:0/22031425 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c01a0ddec0 0x55c01a115060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1585cb740 1 -- 172.21.15.92:0/22031425 >> 172.21.15.92:0/22031425 conn(0x55c01a0d9ca0 msgr2=0x55c01a0dc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1585cb740 1 -- 172.21.15.92:0/22031425 shutdown_connections 2024-04-22T07:04:46.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1585cb740 1 -- 172.21.15.92:0/22031425 wait complete. 2024-04-22T07:04:46.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1585cb740 20 client.5043 trim_cache size 0 max 16384 2024-04-22T07:04:46.214 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:04:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "72eb29c5-e441-491c-9309-4fb4ef16ce27", 2024-04-22T07:04:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2024-04-22T07:04:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4275, 2024-04-22T07:04:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-04-22T07:04:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-04-22T07:04:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-04-22T07:04:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 42.435883994000001, 2024-04-22T07:04:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 20, 2024-04-22T07:04:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 20, 2024-04-22T07:04:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 49.304638253999997 2024-04-22T07:04:46.215 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:04:46.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1559e2640 1 -- 172.21.15.92:0/2421594228 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc13006c920 msgr2=0x7fc13006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1559e2640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc13006c920 0x7fc13006ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc140006fd0 tx=0x7fc140008040 comp rx=0 tx=0).stop 2024-04-22T07:04:46.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1559e2640 1 -- 172.21.15.92:0/2421594228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1500cb8c0 msgr2=0x7fc150157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1559e2640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1500cb8c0 0x7fc150157f40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc13800c3f0 tx=0x7fc13800c8b0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1559e2640 1 -- 172.21.15.92:0/2421594228 shutdown_connections 2024-04-22T07:04:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1559e2640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc13006c920 0x7fc13006ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1559e2640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc150154fa0 0x7fc15014ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1559e2640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1500cb8c0 0x7fc150157f40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1559e2640 1 --2- 172.21.15.92:0/2421594228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1500cb480 0x7fc1500d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1559e2640 1 -- 172.21.15.92:0/2421594228 >> 172.21.15.92:0/2421594228 conn(0x7fc150073930 msgr2=0x7fc1500c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1559e2640 1 -- 172.21.15.92:0/2421594228 shutdown_connections 2024-04-22T07:04:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:04:46.210+0000 7fc1559e2640 1 -- 172.21.15.92:0/2421594228 wait complete. 2024-04-22T07:04:46.219 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-22T07:05:01.220 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-22T07:05:01.220 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.7 2024-04-22T07:05:01.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:05:01.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.329+0000 7f86da68c640 1 Processor -- start 2024-04-22T07:05:01.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.329+0000 7f86da68c640 1 -- start start 2024-04-22T07:05:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86da68c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86d4152b60 0x7f86d40c5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86da68c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86d40632b0 con 0x7f86d40c6aa0 2024-04-22T07:05:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86da68c640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86d40c7740 con 0x7f86d40c9400 2024-04-22T07:05:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86da68c640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f86d40c7a20 con 0x7f86d4152b60 2024-04-22T07:05:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86d8c29640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f86d40c6aa0 0x7f86d4163210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55678/0 (socket says 172.21.15.92:55678) 2024-04-22T07:05:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86d8c29640 1 -- 172.21.15.92:0/627301154 learned_addr learned my addr 172.21.15.92:0/627301154 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:05:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86d37fe640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86d4152b60 0x7f86d40c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 576603280 0 0) 0x7f86d40632b0 con 0x7f86d40c6aa0 2024-04-22T07:05:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86bc003520 con 0x7f86d40c6aa0 2024-04-22T07:05:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4226703474 0 0) 0x7f86bc003520 con 0x7f86d40c6aa0 2024-04-22T07:05:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86d4152b60 msgr2=0x7f86d40c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86d2ffd640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86d4152b60 0x7f86d40c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 >> v1:172.21.15.145:6789/0 conn(0x7f86d40c9400 legacy=0x7f86d4158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d40c8180 con 0x7f86d40c6aa0 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f86cc0019f0 con 0x7f86d40c6aa0 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.330+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 >> v1:172.21.15.92:6789/0 conn(0x7f86d40c6aa0 legacy=0x7f86d4163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d2ffd640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86bc0039e0 0x7f86bc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d2ffd640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86bc004390 0x7f86bc004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d2ffd640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86bc005060 0x7f86bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86d40c7a20 con 0x7f86bc0039e0 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc00bf00 con 0x7f86bc004390 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f86bc00c080 con 0x7f86bc005060 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d3fff640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86bc0039e0 0x7f86bc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d37fe640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86bc004390 0x7f86bc004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d8c29640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86bc005060 0x7f86bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d3fff640 1 -- 172.21.15.92:0/627301154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86bc005060 msgr2=0x7f86bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d3fff640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86bc005060 0x7f86bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d3fff640 1 -- 172.21.15.92:0/627301154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86bc004390 msgr2=0x7f86bc004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d3fff640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86bc004390 0x7f86bc004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d3fff640 1 -- 172.21.15.92:0/627301154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc003520 con 0x7f86bc0039e0 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.331+0000 7f86d8c29640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86bc005060 0x7f86bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.332+0000 7f86d3fff640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86bc0039e0 0x7f86bc003dc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f86c4001830 tx=0x7f86bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66131f214a76937c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.332+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4001430 con 0x7f86bc0039e0 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.332+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c40052f0 con 0x7f86bc0039e0 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.332+0000 7f86d2ffd640 1 -- 172.21.15.92:0/627301154 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4003830 con 0x7f86bc0039e0 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.332+0000 7f86da68c640 1 -- 172.21.15.92:0/627301154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86bc0039e0 msgr2=0x7f86bc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.332+0000 7f86da68c640 1 --2- 172.21.15.92:0/627301154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86bc0039e0 0x7f86bc003dc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f86c4001830 tx=0x7f86bc005830 comp rx=0 tx=0).stop 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.332+0000 7f86d8c29640 1 -- 172.21.15.92:0/627301154 reap_dead start 2024-04-22T07:05:01.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.332+0000 7f86da68c640 1 -- 172.21.15.92:0/627301154 shutdown_connections 2024-04-22T07:05:01.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.332+0000 7f86da68c640 1 -- 172.21.15.92:0/627301154 >> 172.21.15.92:0/627301154 conn(0x7f86d40cc970 msgr2=0x7f86d40ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:01.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.332+0000 7f86da68c640 1 -- 172.21.15.92:0/627301154 shutdown_connections 2024-04-22T07:05:01.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.332+0000 7f86da68c640 1 -- 172.21.15.92:0/627301154 wait complete. 2024-04-22T07:05:01.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.334+0000 7f86da68c640 1 Processor -- start 2024-04-22T07:05:01.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.334+0000 7f86da68c640 1 -- start start 2024-04-22T07:05:01.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.334+0000 7f86da68c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86bc0039e0 0x7f86d41ee850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.334+0000 7f86da68c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86d4150c10 0x7f86d41eed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.334+0000 7f86da68c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d41578d0 0x7f86d4157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.334+0000 7f86da68c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86d40c8a20 con 0x7f86d41578d0 2024-04-22T07:05:01.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.334+0000 7f86da68c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f86d40c8740 con 0x7f86bc0039e0 2024-04-22T07:05:01.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.334+0000 7f86da68c640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f86d4158fa0 con 0x7f86d4150c10 2024-04-22T07:05:01.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d8c29640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d41578d0 0x7f86d4157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d8c29640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d41578d0 0x7f86d4157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51658/0 (socket says 172.21.15.92:51658) 2024-04-22T07:05:01.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d8c29640 1 -- 172.21.15.92:0/3872899886 learned_addr learned my addr 172.21.15.92:0/3872899886 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:01.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d3fff640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86bc0039e0 0x7f86d41ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d37fe640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86d4150c10 0x7f86d41eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d8c29640 1 -- 172.21.15.92:0/3872899886 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86d4150c10 msgr2=0x7f86d41eed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d8c29640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86d4150c10 0x7f86d41eed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d8c29640 1 -- 172.21.15.92:0/3872899886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86bc0039e0 msgr2=0x7f86d41ee850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d8c29640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86bc0039e0 0x7f86d41ee850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d8c29640 1 -- 172.21.15.92:0/3872899886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d41ee350 con 0x7f86d41578d0 2024-04-22T07:05:01.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d37fe640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86d4150c10 0x7f86d41eed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:01.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d8c29640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d41578d0 0x7f86d4157d50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f86d40c6450 tx=0x7f86cc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:01.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d0ff9640 1 -- 172.21.15.92:0/3872899886 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc00ec10 con 0x7f86d41578d0 2024-04-22T07:05:01.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d0ff9640 1 -- 172.21.15.92:0/3872899886 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86cc00edb0 con 0x7f86d41578d0 2024-04-22T07:05:01.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86da68c640 1 -- 172.21.15.92:0/3872899886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d41f22d0 con 0x7f86d41578d0 2024-04-22T07:05:01.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86d0ff9640 1 -- 172.21.15.92:0/3872899886 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc014670 con 0x7f86d41578d0 2024-04-22T07:05:01.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.335+0000 7f86da68c640 1 -- 172.21.15.92:0/3872899886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d4153320 con 0x7f86d41578d0 2024-04-22T07:05:01.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.336+0000 7f86da68c640 1 -- 172.21.15.92:0/3872899886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d4068c40 con 0x7f86d41578d0 2024-04-22T07:05:01.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.337+0000 7f86d0ff9640 1 -- 172.21.15.92:0/3872899886 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86cc014810 con 0x7f86d41578d0 2024-04-22T07:05:01.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.337+0000 7f86d0ff9640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.337+0000 7f86d0ff9640 1 -- 172.21.15.92:0/3872899886 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f86cc08b970 con 0x7f86d41578d0 2024-04-22T07:05:01.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.337+0000 7f86d3fff640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.337+0000 7f86d3fff640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f86c006c8f0 0x7f86c006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f86c4034000 tx=0x7f86c4031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:01.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.341+0000 7f86d0ff9640 1 -- 172.21.15.92:0/3872899886 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86d4068c40 con 0x7f86d41578d0 2024-04-22T07:05:01.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.551+0000 7f86da68c640 1 -- 172.21.15.92:0/3872899886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86d4069930 con 0x7f86d41578d0 2024-04-22T07:05:01.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.552+0000 7f86d0ff9640 1 -- 172.21.15.92:0/3872899886 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6913 (secure 0 0 0) 0x7f86cc059640 con 0x7f86d41578d0 2024-04-22T07:05:01.551 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:05:01.551 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1059009649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1059009649},{"type":"v1","addr":"172.21.15.145:6835","nonce":1059009649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:03.129095+0000","modified":"2024-04-22T07:04:31.038967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4275,"mds_1":4273,"mds_2":4269,"mds_3":4232,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.145:6836/1058270486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1058270486},{"type":"v1","addr":"172.21.15.145:6836","nonce":1058270486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4232":{"gid":4232,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.145:6837/3258302920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3258302920},{"type":"v1","addr":"172.21.15.145:6837","nonce":3258302920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/101798872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":101798872},{"type":"v1","addr":"172.21.15.92:6837","nonce":101798872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6839/3493022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3493022212},{"type":"v1","addr":"172.21.15.92:6839","nonce":3493022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4229,4232,4269,4273,4275]},"id":1}]} 2024-04-22T07:05:01.551 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2024-04-22T07:05:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.553+0000 7f86da68c640 1 -- 172.21.15.92:0/3872899886 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f86c006c8f0 msgr2=0x7f86c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.553+0000 7f86da68c640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f86c006c8f0 0x7f86c006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f86c4034000 tx=0x7f86c4031000 comp rx=0 tx=0).stop 2024-04-22T07:05:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.553+0000 7f86da68c640 1 -- 172.21.15.92:0/3872899886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d41578d0 msgr2=0x7f86d4157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.553+0000 7f86da68c640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d41578d0 0x7f86d4157d50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f86d40c6450 tx=0x7f86cc00e530 comp rx=0 tx=0).stop 2024-04-22T07:05:01.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.553+0000 7f86da68c640 1 -- 172.21.15.92:0/3872899886 shutdown_connections 2024-04-22T07:05:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.553+0000 7f86da68c640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.553+0000 7f86da68c640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d41578d0 0x7f86d4157d50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.553+0000 7f86da68c640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86d4150c10 0x7f86d41eed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.553+0000 7f86da68c640 1 --2- 172.21.15.92:0/3872899886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86bc0039e0 0x7f86d41ee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.553+0000 7f86da68c640 1 -- 172.21.15.92:0/3872899886 >> 172.21.15.92:0/3872899886 conn(0x7f86d40cc970 msgr2=0x7f86d414f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.554+0000 7f86da68c640 1 -- 172.21.15.92:0/3872899886 shutdown_connections 2024-04-22T07:05:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.554+0000 7f86da68c640 1 -- 172.21.15.92:0/3872899886 wait complete. 2024-04-22T07:05:01.563 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.4275 quiesce db / --timeout 60 --expiration 79.7 --await' 2024-04-22T07:05:01.564 DEBUG:teuthology.orchestra.run.smithi092:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4275 quiesce db / --timeout 60 --expiration 79.7 --await 2024-04-22T07:05:01.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.670+0000 7fc380cf2640 1 Processor -- start 2024-04-22T07:05:01.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.670+0000 7fc380cf2640 1 -- start start 2024-04-22T07:05:01.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc380cf2640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc37c151180 0x7fc37c15ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc380cf2640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc37c0c6de0 0x7fc37c1618f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc380cf2640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc37c157ec0 con 0x7fc37c1579f0 2024-04-22T07:05:01.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc380cf2640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fc37c15a560 con 0x7fc37c151180 2024-04-22T07:05:01.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc380cf2640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc37c071910 con 0x7fc37c0c6de0 2024-04-22T07:05:01.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc37a59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc37c151180 0x7fc37c15ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc37a59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc37c151180 0x7fc37c15ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51662/0 (socket says 172.21.15.92:51662) 2024-04-22T07:05:01.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc37a59c640 1 -- 172.21.15.92:0/4163352377 learned_addr learned my addr 172.21.15.92:0/4163352377 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:01.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/4163352377 >> v2:172.21.15.145:3300/0 conn(0x7fc37c0c6de0 0x7fc37c1618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc37a59c640 1 -- 172.21.15.92:0/4163352377 >> v2:172.21.15.145:3300/0 conn(0x7fc37c0c6de0 msgr2=0x7fc37c1618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc37a59c640 1 --2- 172.21.15.92:0/4163352377 >> v2:172.21.15.145:3300/0 conn(0x7fc37c0c6de0 0x7fc37c1618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc37a59c640 1 -- 172.21.15.92:0/4163352377 >> v1:172.21.15.145:6789/0 conn(0x7fc37c1579f0 legacy=0x7fc37c0cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc37a59c640 1 -- 172.21.15.92:0/4163352377 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c071bf0 con 0x7fc37c151180 2024-04-22T07:05:01.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.671+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/4163352377 >> v2:172.21.15.145:3300/0 conn(0x7fc37c0c6de0 0x7fc37c1618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:01.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.672+0000 7fc37a59c640 1 --2- 172.21.15.92:0/4163352377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc37c151180 0x7fc37c15ad40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc37c072cb0 tx=0x7fc36c02e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7d317758857e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:01.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.672+0000 7fc37959a640 1 -- 172.21.15.92:0/4163352377 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c03b070 con 0x7fc37c151180 2024-04-22T07:05:01.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.672+0000 7fc37959a640 1 -- 172.21.15.92:0/4163352377 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc36c036440 con 0x7fc37c151180 2024-04-22T07:05:01.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.672+0000 7fc37959a640 1 -- 172.21.15.92:0/4163352377 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c036810 con 0x7fc37c151180 2024-04-22T07:05:01.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.672+0000 7fc380cf2640 1 -- 172.21.15.92:0/4163352377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc37c151180 msgr2=0x7fc37c15ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.672+0000 7fc380cf2640 1 --2- 172.21.15.92:0/4163352377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc37c151180 0x7fc37c15ad40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc37c072cb0 tx=0x7fc36c02e170 comp rx=0 tx=0).stop 2024-04-22T07:05:01.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.673+0000 7fc380cf2640 1 -- 172.21.15.92:0/4163352377 shutdown_connections 2024-04-22T07:05:01.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.673+0000 7fc380cf2640 1 --2- 172.21.15.92:0/4163352377 >> v2:172.21.15.145:3300/0 conn(0x7fc37c0c6de0 0x7fc37c1618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.673+0000 7fc380cf2640 1 --2- 172.21.15.92:0/4163352377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc37c151180 0x7fc37c15ad40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.673+0000 7fc380cf2640 1 -- 172.21.15.92:0/4163352377 >> 172.21.15.92:0/4163352377 conn(0x7fc37c06d6b0 msgr2=0x7fc37c06da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:01.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.673+0000 7fc380cf2640 1 -- 172.21.15.92:0/4163352377 shutdown_connections 2024-04-22T07:05:01.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.673+0000 7fc380cf2640 1 -- 172.21.15.92:0/4163352377 wait complete. 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.674+0000 7fc380cf2640 1 Processor -- start 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.674+0000 7fc380cf2640 1 -- start start 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc380cf2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc37c0c6de0 0x7fc37c1f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc380cf2640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc37c151180 0x7fc37c14e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc380cf2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc37c1579f0 0x7fc37c1e13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc380cf2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c0728e0 con 0x7fc37c1579f0 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc380cf2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c15a820 con 0x7fc37c0c6de0 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc380cf2640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc37c151560 con 0x7fc37c151180 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc37ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc37c1579f0 0x7fc37c1e13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc37ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc37c1579f0 0x7fc37c1e13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51668/0 (socket says 172.21.15.92:51668) 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc37ad9d640 1 -- 172.21.15.92:0/1933728742 learned_addr learned my addr 172.21.15.92:0/1933728742 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc379d9b640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc37c151180 0x7fc37c14e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc37a59c640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc37c0c6de0 0x7fc37c1f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc37ad9d640 1 -- 172.21.15.92:0/1933728742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc37c151180 msgr2=0x7fc37c14e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc37c151180 0x7fc37c14e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc37ad9d640 1 -- 172.21.15.92:0/1933728742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc37c0c6de0 msgr2=0x7fc37c1f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc37c0c6de0 0x7fc37c1f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc37ad9d640 1 -- 172.21.15.92:0/1933728742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c1e0f10 con 0x7fc37c1579f0 2024-04-22T07:05:01.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc37c1579f0 0x7fc37c1e13c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc37400cf20 tx=0x7fc37400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:01.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc36affd640 1 -- 172.21.15.92:0/1933728742 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37401a070 con 0x7fc37c1579f0 2024-04-22T07:05:01.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc36affd640 1 -- 172.21.15.92:0/1933728742 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37400d840 con 0x7fc37c1579f0 2024-04-22T07:05:01.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc380cf2640 1 -- 172.21.15.92:0/1933728742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c0c82a0 con 0x7fc37c1579f0 2024-04-22T07:05:01.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.675+0000 7fc36affd640 1 -- 172.21.15.92:0/1933728742 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37400dc30 con 0x7fc37c1579f0 2024-04-22T07:05:01.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.676+0000 7fc380cf2640 1 -- 172.21.15.92:0/1933728742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c0c8840 con 0x7fc37c1579f0 2024-04-22T07:05:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.677+0000 7fc36affd640 1 -- 172.21.15.92:0/1933728742 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc37401d070 con 0x7fc37c1579f0 2024-04-22T07:05:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.677+0000 7fc36affd640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc35c06c8f0 0x7fc35c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.677+0000 7fc36affd640 1 -- 172.21.15.92:0/1933728742 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc37408c390 con 0x7fc37c1579f0 2024-04-22T07:05:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.677+0000 7fc37a59c640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc35c06c8f0 0x7fc35c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.678+0000 7fc37a59c640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc35c06c8f0 0x7fc35c06edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc36c038000 tx=0x7fc36c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.678+0000 7fc3839be740 1 Processor -- start 2024-04-22T07:05:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.678+0000 7fc3839be740 1 -- start start 2024-04-22T07:05:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc3839be740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c46308f200 0x55c46308f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc3839be740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630d9080 0x55c46308fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc3839be740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308dc60 0x55c4630ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc3839be740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c462ff00b0 con 0x55c46308f200 2024-04-22T07:05:01.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc3839be740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c462fa1680 con 0x55c4630d9080 2024-04-22T07:05:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc3839be740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55c462fadc60 con 0x55c46308dc60 2024-04-22T07:05:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc37a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c46308f200 0x55c46308f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc37a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c46308f200 0x55c46308f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51676/0 (socket says 172.21.15.92:51676) 2024-04-22T07:05:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc37a59c640 1 -- 172.21.15.92:0/1061298684 learned_addr learned my addr 172.21.15.92:0/1061298684 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/1061298684 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308dc60 0x55c4630ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc379d9b640 1 --2- 172.21.15.92:0/1061298684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630d9080 0x55c46308fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc37a59c640 1 -- 172.21.15.92:0/1061298684 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308dc60 msgr2=0x55c4630ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc37a59c640 1 --2- 172.21.15.92:0/1061298684 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308dc60 0x55c4630ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc37a59c640 1 -- 172.21.15.92:0/1061298684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630d9080 msgr2=0x55c46308fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc37a59c640 1 --2- 172.21.15.92:0/1061298684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630d9080 0x55c46308fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc37a59c640 1 -- 172.21.15.92:0/1061298684 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c462dd2c90 con 0x55c46308f200 2024-04-22T07:05:01.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.679+0000 7fc379d9b640 1 --2- 172.21.15.92:0/1061298684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630d9080 0x55c46308fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.680+0000 7fc37a59c640 1 --2- 172.21.15.92:0/1061298684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c46308f200 0x55c46308f5e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc36c039040 tx=0x7fc36c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b2a7a13f6e7927 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.680+0000 7fc35bfff640 1 -- 172.21.15.92:0/1061298684 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c051070 con 0x55c46308f200 2024-04-22T07:05:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.680+0000 7fc35bfff640 1 -- 172.21.15.92:0/1061298684 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc36c04c470 con 0x55c46308f200 2024-04-22T07:05:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.680+0000 7fc35bfff640 1 -- 172.21.15.92:0/1061298684 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c03b040 con 0x55c46308f200 2024-04-22T07:05:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.680+0000 7fc3839be740 1 -- 172.21.15.92:0/1061298684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c46308f200 msgr2=0x55c46308f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.680+0000 7fc3839be740 1 --2- 172.21.15.92:0/1061298684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c46308f200 0x55c46308f5e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc36c039040 tx=0x7fc36c030000 comp rx=0 tx=0).stop 2024-04-22T07:05:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.681+0000 7fc3839be740 1 -- 172.21.15.92:0/1061298684 shutdown_connections 2024-04-22T07:05:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.681+0000 7fc3839be740 1 --2- 172.21.15.92:0/1061298684 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308dc60 0x55c4630ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.681+0000 7fc3839be740 1 --2- 172.21.15.92:0/1061298684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630d9080 0x55c46308fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.681+0000 7fc3839be740 1 --2- 172.21.15.92:0/1061298684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c46308f200 0x55c46308f5e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.681+0000 7fc3839be740 1 -- 172.21.15.92:0/1061298684 >> 172.21.15.92:0/1061298684 conn(0x55c46308bc00 msgr2=0x55c46308e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.681+0000 7fc3839be740 1 -- 172.21.15.92:0/1061298684 shutdown_connections 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.681+0000 7fc3839be740 1 -- 172.21.15.92:0/1061298684 wait complete. 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.681+0000 7fc3839be740 1 -- start start 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.681+0000 7fc3839be740 1 Processor -- start 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc3839be740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c46308f200 0x55c4630b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc3839be740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c4630d9080 0x55c4630b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc3839be740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 0x55c4630d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc3839be740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c462ff3400 con 0x55c4630e1c60 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc3839be740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c462faf840 con 0x55c46308f200 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc3839be740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55c462ff8720 con 0x55c4630d9080 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc3839be740 10 client.0 _ll_register_callbacks cb 0x55c463000ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc37ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 0x55c4630d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc37ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 0x55c4630d20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51692/0 (socket says 172.21.15.92:51692) 2024-04-22T07:05:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc37ad9d640 1 -- 172.21.15.92:0/3345294439 learned_addr learned my addr 172.21.15.92:0/3345294439 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc379d9b640 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c4630d9080 0x55c4630b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc37a59c640 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c46308f200 0x55c4630b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc37ad9d640 1 -- 172.21.15.92:0/3345294439 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c4630d9080 msgr2=0x55c4630b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c4630d9080 0x55c4630b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc37ad9d640 1 -- 172.21.15.92:0/3345294439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c46308f200 msgr2=0x55c4630b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c46308f200 0x55c4630b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc37ad9d640 1 -- 172.21.15.92:0/3345294439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c462e33420 con 0x55c4630e1c60 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.682+0000 7fc379d9b640 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c4630d9080 0x55c4630b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 0x55c4630d20b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc37400acd0 tx=0x7fc374056410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362b99a37839cea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3597fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3597fa640 1 -- 172.21.15.92:0/3345294439 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37405e0d0 con 0x55c4630e1c60 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3597fa640 1 -- 172.21.15.92:0/3345294439 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc37408c0b0 con 0x55c4630e1c60 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3839be740 1 -- 172.21.15.92:0/3345294439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc36c04c470 con 0x55c4630e1c60 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3597fa640 1 -- 172.21.15.92:0/3345294439 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc374062790 con 0x55c4630e1c60 2024-04-22T07:05:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3839be740 10 client.5061 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3839be740 1 -- 172.21.15.92:0/3345294439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55c462cc6580 con 0x55c4630e1c60 2024-04-22T07:05:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3597fa640 1 -- 172.21.15.92:0/3345294439 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc36c04c470 con 0x55c4630e1c60 2024-04-22T07:05:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3597fa640 1 -- 172.21.15.92:0/3345294439 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fc374059540 con 0x55c4630e1c60 2024-04-22T07:05:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3839be740 10 client.5061 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3839be740 10 client.5061 resolve_mds: validated gid 4275 aka daemon mds.e 2024-04-22T07:05:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3839be740 20 client.5061 populate_metadata read hostname 'smithi092' 2024-04-22T07:05:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3839be740 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c4630cca90 0x55c4630cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3839be740 4 client.5061 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc3839be740 1 -- 172.21.15.92:0/3345294439 --> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c463001b10 con 0x55c4630cca90 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.683+0000 7fc37a59c640 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c4630cca90 0x55c4630cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc37a59c640 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c4630cca90 0x55c4630cce70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ffb16403af9dfca server_cookie=765bef9b42329dc in_seq=0 out_seq=0 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3597fa640 10 client.5061 ms_handle_connect on v2:172.21.15.92:6838/3493022212 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3597fa640 1 -- 172.21.15.92:0/3345294439 <== mds.0 v2:172.21.15.92:6838/3493022212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c463001b10 con 0x55c4630cca90 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3597fa640 10 client.5061 handle_command_reply: tid=0 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3839be740 1 client.5061 shutdown 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3839be740 1 -- 172.21.15.92:0/3345294439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 msgr2=0x55c4630d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3839be740 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 0x55c4630d20b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc37400acd0 tx=0x7fc374056410 comp rx=0 tx=0).stop 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3839be740 1 -- 172.21.15.92:0/3345294439 shutdown_connections 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3839be740 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c4630cca90 0x55c4630cce70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3839be740 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 0x55c4630d20b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3839be740 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c4630d9080 0x55c4630b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3839be740 1 --2- 172.21.15.92:0/3345294439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c46308f200 0x55c4630b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3597fa640 0 client.5061 ms_handle_reset on v2:172.21.15.92:6838/3493022212 2024-04-22T07:05:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3839be740 1 -- 172.21.15.92:0/3345294439 >> 172.21.15.92:0/3345294439 conn(0x55c46308dc60 msgr2=0x55c463088940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3839be740 1 -- 172.21.15.92:0/3345294439 shutdown_connections 2024-04-22T07:05:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.684+0000 7fc3839be740 1 -- 172.21.15.92:0/3345294439 wait complete. 2024-04-22T07:05:01.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.685+0000 7fc3839be740 20 client.5061 trim_cache size 0 max 16384 2024-04-22T07:05:01.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.702+0000 7fc3839be740 1 Processor -- start 2024-04-22T07:05:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.702+0000 7fc3839be740 1 -- start start 2024-04-22T07:05:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.702+0000 7fc3839be740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308f200 0x55c4630cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.702+0000 7fc3839be740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630d9080 0x55c46308f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.702+0000 7fc3839be740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 0x55c46308fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.702+0000 7fc3839be740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1fe010 con 0x55c4630e1c60 2024-04-22T07:05:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.702+0000 7fc3839be740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1fc540 con 0x55c4630d9080 2024-04-22T07:05:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.702+0000 7fc3839be740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc37c1fd940 con 0x55c46308f200 2024-04-22T07:05:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.702+0000 7fc37ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 0x55c46308fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.702+0000 7fc37ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 0x55c46308fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51696/0 (socket says 172.21.15.92:51696) 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.702+0000 7fc37ad9d640 1 -- 172.21.15.92:0/2724405290 learned_addr learned my addr 172.21.15.92:0/2724405290 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc37a59c640 1 --2- 172.21.15.92:0/2724405290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308f200 0x55c4630cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc379d9b640 1 --2- 172.21.15.92:0/2724405290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630d9080 0x55c46308f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc37ad9d640 1 -- 172.21.15.92:0/2724405290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308f200 msgr2=0x55c4630cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/2724405290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308f200 0x55c4630cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc37ad9d640 1 -- 172.21.15.92:0/2724405290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630d9080 msgr2=0x55c46308f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/2724405290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630d9080 0x55c46308f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc37ad9d640 1 -- 172.21.15.92:0/2724405290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4630b52d0 con 0x55c4630e1c60 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/2724405290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 0x55c46308fb20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc374062de0 tx=0x7fc37405c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969d8dbd29155751 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc35b7fe640 1 -- 172.21.15.92:0/2724405290 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374062a20 con 0x55c4630e1c60 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc35b7fe640 1 -- 172.21.15.92:0/2724405290 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37400ddd0 con 0x55c4630e1c60 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc35b7fe640 1 -- 172.21.15.92:0/2724405290 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37408c0b0 con 0x55c4630e1c60 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc3839be740 1 -- 172.21.15.92:0/2724405290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 msgr2=0x55c46308fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc3839be740 1 --2- 172.21.15.92:0/2724405290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 0x55c46308fb20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc374062de0 tx=0x7fc37405c000 comp rx=0 tx=0).stop 2024-04-22T07:05:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc3839be740 1 -- 172.21.15.92:0/2724405290 shutdown_connections 2024-04-22T07:05:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.703+0000 7fc3839be740 1 --2- 172.21.15.92:0/2724405290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630e1c60 0x55c46308fb20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.704+0000 7fc3839be740 1 --2- 172.21.15.92:0/2724405290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630d9080 0x55c46308f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.704+0000 7fc3839be740 1 --2- 172.21.15.92:0/2724405290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308f200 0x55c4630cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.704+0000 7fc3839be740 1 -- 172.21.15.92:0/2724405290 >> 172.21.15.92:0/2724405290 conn(0x55c46308dc60 msgr2=0x55c4630b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.704+0000 7fc3839be740 1 -- 172.21.15.92:0/2724405290 shutdown_connections 2024-04-22T07:05:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.704+0000 7fc3839be740 1 -- 172.21.15.92:0/2724405290 wait complete. 2024-04-22T07:05:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.704+0000 7fc3839be740 1 -- start start 2024-04-22T07:05:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.704+0000 7fc3839be740 1 Processor -- start 2024-04-22T07:05:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc3839be740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308f200 0x55c4630d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc3839be740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630d9080 0x55c4630f4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc3839be740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630e1c60 0x55c46314c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc3839be740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1fd270 con 0x55c4630d9080 2024-04-22T07:05:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc3839be740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1fcba0 con 0x55c4630e1c60 2024-04-22T07:05:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc3839be740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc37c14f9f0 con 0x55c46308f200 2024-04-22T07:05:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc3839be740 10 client.0 _ll_register_callbacks cb 0x7fc374004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc379d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630d9080 0x55c4630f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc379d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630d9080 0x55c4630f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51700/0 (socket says 172.21.15.92:51700) 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc379d9b640 1 -- 172.21.15.92:0/825169992 learned_addr learned my addr 172.21.15.92:0/825169992 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc37ad9d640 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630e1c60 0x55c46314c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc37a59c640 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308f200 0x55c4630d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc379d9b640 1 -- 172.21.15.92:0/825169992 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308f200 msgr2=0x55c4630d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc379d9b640 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308f200 0x55c4630d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc379d9b640 1 -- 172.21.15.92:0/825169992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630e1c60 msgr2=0x55c46314c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc379d9b640 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630e1c60 0x55c46314c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc379d9b640 1 -- 172.21.15.92:0/825169992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4630d7800 con 0x55c4630d9080 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc37a59c640 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308f200 0x55c4630d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc379d9b640 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630d9080 0x55c4630f4190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0c30b0 tx=0x7fc370009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548b4531b4abf3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.705+0000 7fc3597fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:05:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3597fa640 1 -- 172.21.15.92:0/825169992 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37001b070 con 0x55c4630d9080 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3597fa640 1 -- 172.21.15.92:0/825169992 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc370004020 con 0x55c4630d9080 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3839be740 1 -- 172.21.15.92:0/825169992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc37400ddd0 con 0x55c4630d9080 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3597fa640 1 -- 172.21.15.92:0/825169992 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc370026050 con 0x55c4630d9080 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3839be740 10 client.5067 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3839be740 1 -- 172.21.15.92:0/825169992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55c4630e51e0 con 0x55c4630d9080 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3597fa640 1 -- 172.21.15.92:0/825169992 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37400ddd0 con 0x55c4630d9080 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3597fa640 1 -- 172.21.15.92:0/825169992 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fc370029020 con 0x55c4630d9080 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3839be740 10 client.5067 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3839be740 10 client.5067 resolve_mds: validated gid 4275 aka daemon mds.e 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3839be740 20 client.5067 populate_metadata read hostname 'smithi092' 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3839be740 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c4630b3700 0x55c46314cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3839be740 4 client.5067 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.7, "await": true}] 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3839be740 1 -- 172.21.15.92:0/825169992 --> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.7, "await": true}) -- 0x55c462ffb980 con 0x55c4630b3700 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc37a59c640 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c4630b3700 0x55c46314cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc3597fa640 10 client.5067 ms_handle_connect on v2:172.21.15.92:6838/3493022212 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.706+0000 7fc37a59c640 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c4630b3700 0x55c46314cca0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24971531e3c05a7 server_cookie=2bdbc698a42821de in_seq=0 out_seq=0 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3597fa640 1 -- 172.21.15.92:0/825169992 <== mds.0 v2:172.21.15.92:6838/3493022212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+587 (crc 0 0 0) 0x55c462ffb980 con 0x55c4630b3700 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3597fa640 10 client.5067 handle_command_reply: tid=0 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3839be740 1 client.5067 shutdown 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3839be740 1 -- 172.21.15.92:0/825169992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630d9080 msgr2=0x55c4630f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3839be740 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630d9080 0x55c4630f4190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0c30b0 tx=0x7fc370009f70 comp rx=0 tx=0).stop 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3839be740 1 -- 172.21.15.92:0/825169992 shutdown_connections 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3839be740 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x55c4630b3700 0x55c46314cca0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3839be740 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c4630e1c60 0x55c46314c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3839be740 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4630d9080 0x55c4630f4190 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3839be740 1 --2- 172.21.15.92:0/825169992 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55c46308f200 0x55c4630d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3597fa640 0 client.5067 ms_handle_reset on v2:172.21.15.92:6838/3493022212 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3839be740 1 -- 172.21.15.92:0/825169992 >> 172.21.15.92:0/825169992 conn(0x55c46308dc60 msgr2=0x55c4630d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3839be740 1 -- 172.21.15.92:0/825169992 shutdown_connections 2024-04-22T07:05:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3839be740 1 -- 172.21.15.92:0/825169992 wait complete. 2024-04-22T07:05:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.709+0000 7fc3839be740 20 client.5067 trim_cache size 0 max 16384 2024-04-22T07:05:01.713 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:05:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 17, 2024-04-22T07:05:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "leader": 4275, 2024-04-22T07:05:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "set_version": 2, 2024-04-22T07:05:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "sets": { 2024-04-22T07:05:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "2853aa63": { 2024-04-22T07:05:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "version": 2, 2024-04-22T07:05:01.713 INFO:teuthology.orchestra.run.smithi092.stdout: "age_ref": 0.0, 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "state": { 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "QUIESCED", 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "timeout": 60.0, 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "expiration": 79.7, 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "members": { 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "file:/": { 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "excluded": false, 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "state": { 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "QUIESCED", 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:01.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:01.715 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:05:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.710+0000 7fc380cf2640 1 -- 172.21.15.92:0/1933728742 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc35c06c8f0 msgr2=0x7fc35c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.710+0000 7fc380cf2640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc35c06c8f0 0x7fc35c06edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc36c038000 tx=0x7fc36c008040 comp rx=0 tx=0).stop 2024-04-22T07:05:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.710+0000 7fc380cf2640 1 -- 172.21.15.92:0/1933728742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc37c1579f0 msgr2=0x7fc37c1e13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.710+0000 7fc380cf2640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc37c1579f0 0x7fc37c1e13c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc37400cf20 tx=0x7fc37400a7d0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.710+0000 7fc380cf2640 1 -- 172.21.15.92:0/1933728742 shutdown_connections 2024-04-22T07:05:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.710+0000 7fc380cf2640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc35c06c8f0 0x7fc35c06edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.710+0000 7fc380cf2640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc37c1579f0 0x7fc37c1e13c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.710+0000 7fc380cf2640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc37c151180 0x7fc37c14e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.710+0000 7fc380cf2640 1 --2- 172.21.15.92:0/1933728742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc37c0c6de0 0x7fc37c1f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.710+0000 7fc380cf2640 1 -- 172.21.15.92:0/1933728742 >> 172.21.15.92:0/1933728742 conn(0x7fc37c06d6b0 msgr2=0x7fc37c14be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.710+0000 7fc380cf2640 1 -- 172.21.15.92:0/1933728742 shutdown_connections 2024-04-22T07:05:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:01.710+0000 7fc380cf2640 1 -- 172.21.15.92:0/1933728742 wait complete. 2024-04-22T07:05:01.719 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: 2853aa63, quiesce took 0.5 seconds. Will release after: 19.7 2024-04-22T07:05:05.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:05:05.079 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:05:21.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:05:21.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.528+0000 7f457005c640 1 Processor -- start 2024-04-22T07:05:21.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.528+0000 7f457005c640 1 -- start start 2024-04-22T07:05:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f457005c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4568152940 0x7f45680c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f457005c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f456806b970 0x7f4568158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f457005c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45680c9b00 0x7f4568070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f457005c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f456806c740 con 0x7f4568152940 2024-04-22T07:05:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f457005c640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f45680c6cf0 con 0x7f456806b970 2024-04-22T07:05:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f457005c640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f45680c8580 con 0x7f45680c9b00 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f456ddf8640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4568152940 0x7f45680c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f456ddf8640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4568152940 0x7f45680c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38832/0 (socket says 172.21.15.92:38832) 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f456ddf8640 1 -- 172.21.15.92:0/3625093796 learned_addr learned my addr 172.21.15.92:0/3625093796 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f456e5f9640 1 --2- 172.21.15.92:0/3625093796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45680c9b00 0x7f4568070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f456ddf8640 1 -- 172.21.15.92:0/3625093796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45680c9b00 msgr2=0x7f4568070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f456d5f7640 1 --2- 172.21.15.92:0/3625093796 >> v2:172.21.15.145:3300/0 conn(0x7f456806b970 0x7f4568158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f456ddf8640 1 --2- 172.21.15.92:0/3625093796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45680c9b00 0x7f4568070ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f456ddf8640 1 -- 172.21.15.92:0/3625093796 >> v2:172.21.15.145:3300/0 conn(0x7f456806b970 msgr2=0x7f4568158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f456ddf8640 1 --2- 172.21.15.92:0/3625093796 >> v2:172.21.15.145:3300/0 conn(0x7f456806b970 0x7f4568158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f456ddf8640 1 -- 172.21.15.92:0/3625093796 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45680c6f90 con 0x7f4568152940 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f456e5f9640 1 --2- 172.21.15.92:0/3625093796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45680c9b00 0x7f4568070ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.529+0000 7f456d5f7640 1 --2- 172.21.15.92:0/3625093796 >> v2:172.21.15.145:3300/0 conn(0x7f456806b970 0x7f4568158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.530+0000 7f456ddf8640 1 --2- 172.21.15.92:0/3625093796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4568152940 0x7f45680c7e40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f45680c7720 tx=0x7f456402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7eb9cd41b7a3cc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:21.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.530+0000 7f456cdf6640 1 -- 172.21.15.92:0/3625093796 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564035020 con 0x7f4568152940 2024-04-22T07:05:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.530+0000 7f456cdf6640 1 -- 172.21.15.92:0/3625093796 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4564031870 con 0x7f4568152940 2024-04-22T07:05:21.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.530+0000 7f456cdf6640 1 -- 172.21.15.92:0/3625093796 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564031c40 con 0x7f4568152940 2024-04-22T07:05:21.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.531+0000 7f457005c640 1 -- 172.21.15.92:0/3625093796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4568152940 msgr2=0x7f45680c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.531+0000 7f457005c640 1 --2- 172.21.15.92:0/3625093796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4568152940 0x7f45680c7e40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f45680c7720 tx=0x7f456402b910 comp rx=0 tx=0).stop 2024-04-22T07:05:21.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.531+0000 7f457005c640 1 -- 172.21.15.92:0/3625093796 shutdown_connections 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.531+0000 7f457005c640 1 --2- 172.21.15.92:0/3625093796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45680c9b00 0x7f4568070ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.531+0000 7f457005c640 1 --2- 172.21.15.92:0/3625093796 >> v2:172.21.15.145:3300/0 conn(0x7f456806b970 0x7f4568158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.531+0000 7f457005c640 1 --2- 172.21.15.92:0/3625093796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4568152940 0x7f45680c7e40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.531+0000 7f457005c640 1 -- 172.21.15.92:0/3625093796 >> 172.21.15.92:0/3625093796 conn(0x7f45680c9f40 msgr2=0x7f45680ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.531+0000 7f457005c640 1 -- 172.21.15.92:0/3625093796 shutdown_connections 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.531+0000 7f457005c640 1 -- 172.21.15.92:0/3625093796 wait complete. 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.532+0000 7f457005c640 1 Processor -- start 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.532+0000 7f457005c640 1 -- start start 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f457005c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f456806b970 0x7f456814d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f457005c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45680c9b00 0x7f4568155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f457005c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4568152940 0x7f4568148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f457005c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4568152d20 con 0x7f4568152940 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f457005c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4568158f40 con 0x7f45680c9b00 2024-04-22T07:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f457005c640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f456814d730 con 0x7f456806b970 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f456e5f9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4568152940 0x7f4568148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f456e5f9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4568152940 0x7f4568148f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38838/0 (socket says 172.21.15.92:38838) 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f456e5f9640 1 -- 172.21.15.92:0/290600814 learned_addr learned my addr 172.21.15.92:0/290600814 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f456d5f7640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45680c9b00 0x7f4568155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f456ddf8640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f456806b970 0x7f456814d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f456e5f9640 1 -- 172.21.15.92:0/290600814 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f456806b970 msgr2=0x7f456814d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f456e5f9640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f456806b970 0x7f456814d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f456e5f9640 1 -- 172.21.15.92:0/290600814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45680c9b00 msgr2=0x7f4568155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f456e5f9640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45680c9b00 0x7f4568155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f456e5f9640 1 -- 172.21.15.92:0/290600814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4568153e10 con 0x7f4568152940 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f456d5f7640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45680c9b00 0x7f4568155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f456e5f9640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4568152940 0x7f4568148f40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f456000cea0 tx=0x7f456000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f455e7fc640 1 -- 172.21.15.92:0/290600814 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560003b90 con 0x7f4568152940 2024-04-22T07:05:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f455e7fc640 1 -- 172.21.15.92:0/290600814 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4560003d50 con 0x7f4568152940 2024-04-22T07:05:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f457005c640 1 -- 172.21.15.92:0/290600814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45681540f0 con 0x7f4568152940 2024-04-22T07:05:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.533+0000 7f455e7fc640 1 -- 172.21.15.92:0/290600814 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560004260 con 0x7f4568152940 2024-04-22T07:05:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.534+0000 7f457005c640 1 -- 172.21.15.92:0/290600814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45681546b0 con 0x7f4568152940 2024-04-22T07:05:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.535+0000 7f457005c640 1 -- 172.21.15.92:0/290600814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4568069930 con 0x7f4568152940 2024-04-22T07:05:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.535+0000 7f455e7fc640 1 -- 172.21.15.92:0/290600814 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4560004400 con 0x7f4568152940 2024-04-22T07:05:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.535+0000 7f455e7fc640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f454c06c920 0x7f454c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.536+0000 7f456ddf8640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f454c06c920 0x7f454c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.536+0000 7f455e7fc640 1 -- 172.21.15.92:0/290600814 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f456008c490 con 0x7f4568152940 2024-04-22T07:05:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.536+0000 7f456ddf8640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f454c06c920 0x7f454c06ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4564036000 tx=0x7f4564008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.540+0000 7f455e7fc640 1 -- 172.21.15.92:0/290600814 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f456005a8d0 con 0x7f4568152940 2024-04-22T07:05:21.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.752+0000 7f457005c640 1 -- 172.21.15.92:0/290600814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45680cc110 con 0x7f4568152940 2024-04-22T07:05:21.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.752+0000 7f455e7fc640 1 -- 172.21.15.92:0/290600814 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6913 (secure 0 0 0) 0x7f456005a1d0 con 0x7f4568152940 2024-04-22T07:05:21.751 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:05:21.751 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1059009649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1059009649},{"type":"v1","addr":"172.21.15.145:6835","nonce":1059009649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:03.129095+0000","modified":"2024-04-22T07:04:31.038967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4275,"mds_1":4273,"mds_2":4269,"mds_3":4232,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.145:6836/1058270486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1058270486},{"type":"v1","addr":"172.21.15.145:6836","nonce":1058270486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4232":{"gid":4232,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.145:6837/3258302920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3258302920},{"type":"v1","addr":"172.21.15.145:6837","nonce":3258302920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/101798872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":101798872},{"type":"v1","addr":"172.21.15.92:6837","nonce":101798872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6839/3493022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3493022212},{"type":"v1","addr":"172.21.15.92:6839","nonce":3493022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4229,4232,4269,4273,4275]},"id":1}]} 2024-04-22T07:05:21.751 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2024-04-22T07:05:21.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.754+0000 7f457005c640 1 -- 172.21.15.92:0/290600814 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f454c06c920 msgr2=0x7f454c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.754+0000 7f457005c640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f454c06c920 0x7f454c06ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4564036000 tx=0x7f4564008030 comp rx=0 tx=0).stop 2024-04-22T07:05:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.754+0000 7f457005c640 1 -- 172.21.15.92:0/290600814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4568152940 msgr2=0x7f4568148f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.754+0000 7f457005c640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4568152940 0x7f4568148f40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f456000cea0 tx=0x7f456000cf90 comp rx=0 tx=0).stop 2024-04-22T07:05:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.754+0000 7f457005c640 1 -- 172.21.15.92:0/290600814 shutdown_connections 2024-04-22T07:05:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.754+0000 7f457005c640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f454c06c920 0x7f454c06ede0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.754+0000 7f457005c640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4568152940 0x7f4568148f40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.754+0000 7f457005c640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45680c9b00 0x7f4568155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.754+0000 7f457005c640 1 --2- 172.21.15.92:0/290600814 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f456806b970 0x7f456814d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.754+0000 7f457005c640 1 -- 172.21.15.92:0/290600814 >> 172.21.15.92:0/290600814 conn(0x7f45680c9f40 msgr2=0x7f4568159150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.754+0000 7f457005c640 1 -- 172.21.15.92:0/290600814 shutdown_connections 2024-04-22T07:05:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.754+0000 7f457005c640 1 -- 172.21.15.92:0/290600814 wait complete. 2024-04-22T07:05:21.763 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.4275 quiesce db --set-id 2853aa63 --release --await' 2024-04-22T07:05:21.763 DEBUG:teuthology.orchestra.run.smithi092:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4275 quiesce db --set-id 2853aa63 --release --await 2024-04-22T07:05:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.868+0000 7f158838e640 1 Processor -- start 2024-04-22T07:05:21.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.868+0000 7f158838e640 1 -- start start 2024-04-22T07:05:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f158838e640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f158006b970 0x7f15800676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f158838e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f158015bb30 con 0x7f1580067bf0 2024-04-22T07:05:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f158838e640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1580068140 con 0x7f1580154d40 2024-04-22T07:05:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f158838e640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f158015b6a0 con 0x7f158006b970 2024-04-22T07:05:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f158692b640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1580067bf0 0x7f15800c8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40820/0 (socket says 172.21.15.92:40820) 2024-04-22T07:05:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f158692b640 1 -- 172.21.15.92:0/3671846888 learned_addr learned my addr 172.21.15.92:0/3671846888 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:05:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585929640 1 --2- 172.21.15.92:0/3671846888 >> v2:172.21.15.145:3300/0 conn(0x7f158006b970 0x7f15800676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170523909 0 0) 0x7f158015bb30 con 0x7f1580067bf0 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f156c003520 con 0x7f1580067bf0 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1297072137 0 0) 0x7f156c003520 con 0x7f1580067bf0 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 >> v2:172.21.15.145:3300/0 conn(0x7f158006b970 msgr2=0x7f15800676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 --2- 172.21.15.92:0/3671846888 >> v2:172.21.15.145:3300/0 conn(0x7f158006b970 0x7f15800676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 >> v1:172.21.15.145:6789/0 conn(0x7f1580154d40 legacy=0x7f158006c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158015ad70 con 0x7f1580067bf0 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f15780019f0 con 0x7f1580067bf0 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 >> v1:172.21.15.92:6789/0 conn(0x7f1580067bf0 legacy=0x7f15800c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 --2- 172.21.15.92:0/3671846888 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f156c0039b0 0x7f156c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 --2- 172.21.15.92:0/3671846888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c004800 0x7f156c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 --2- 172.21.15.92:0/3671846888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f156c0051c0 0x7f156c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f158015b6a0 con 0x7f156c004800 2024-04-22T07:05:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f156c00c070 con 0x7f156c0051c0 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f156c00c270 con 0x7f156c0039b0 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.869+0000 7f1585929640 1 --2- 172.21.15.92:0/3671846888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c004800 0x7f156c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.870+0000 7f158612a640 1 --2- 172.21.15.92:0/3671846888 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f156c0039b0 0x7f156c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.870+0000 7f158692b640 1 --2- 172.21.15.92:0/3671846888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f156c0051c0 0x7f156c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.870+0000 7f1585929640 1 -- 172.21.15.92:0/3671846888 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f156c0039b0 msgr2=0x7f156c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.870+0000 7f1585929640 1 --2- 172.21.15.92:0/3671846888 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f156c0039b0 0x7f156c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.870+0000 7f1585929640 1 -- 172.21.15.92:0/3671846888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f156c0051c0 msgr2=0x7f156c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.870+0000 7f1585929640 1 --2- 172.21.15.92:0/3671846888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f156c0051c0 0x7f156c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.870+0000 7f158692b640 1 -- 172.21.15.92:0/3671846888 reap_dead start 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.870+0000 7f1585929640 1 -- 172.21.15.92:0/3671846888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156c003520 con 0x7f156c004800 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.870+0000 7f1585929640 1 --2- 172.21.15.92:0/3671846888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c004800 0x7f156c004c80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1570009ac0 tx=0x7f156c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac1c65ce4885ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.871+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570039070 con 0x7f156c004800 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.871+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f157002c6b0 con 0x7f156c004800 2024-04-22T07:05:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.871+0000 7f1585128640 1 -- 172.21.15.92:0/3671846888 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570009690 con 0x7f156c004800 2024-04-22T07:05:21.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.871+0000 7f158838e640 1 -- 172.21.15.92:0/3671846888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c004800 msgr2=0x7f156c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.871+0000 7f158838e640 1 --2- 172.21.15.92:0/3671846888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c004800 0x7f156c004c80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1570009ac0 tx=0x7f156c005830 comp rx=0 tx=0).stop 2024-04-22T07:05:21.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.871+0000 7f158838e640 1 -- 172.21.15.92:0/3671846888 shutdown_connections 2024-04-22T07:05:21.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.871+0000 7f158838e640 1 --2- 172.21.15.92:0/3671846888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c004800 0x7f156c004c80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.871+0000 7f158838e640 1 -- 172.21.15.92:0/3671846888 >> 172.21.15.92:0/3671846888 conn(0x7f15800cd440 msgr2=0x7f15800cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:21.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.871+0000 7f158838e640 1 -- 172.21.15.92:0/3671846888 shutdown_connections 2024-04-22T07:05:21.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.871+0000 7f158838e640 1 -- 172.21.15.92:0/3671846888 wait complete. 2024-04-22T07:05:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.872+0000 7f158838e640 1 Processor -- start 2024-04-22T07:05:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.872+0000 7f158838e640 1 -- start start 2024-04-22T07:05:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158838e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c004800 0x7f1580159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158838e640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15801f4340 0x7f158014cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158838e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f158014d450 0x7f15801519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158838e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1580155120 con 0x7f156c004800 2024-04-22T07:05:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158838e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f158015b330 con 0x7f158014d450 2024-04-22T07:05:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158838e640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f15801552a0 con 0x7f15801f4340 2024-04-22T07:05:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158612a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c004800 0x7f1580159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158612a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c004800 0x7f1580159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38860/0 (socket says 172.21.15.92:38860) 2024-04-22T07:05:21.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158612a640 1 -- 172.21.15.92:0/1221559200 learned_addr learned my addr 172.21.15.92:0/1221559200 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158692b640 1 --2- 172.21.15.92:0/1221559200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f158014d450 0x7f15801519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f1585929640 1 --2- 172.21.15.92:0/1221559200 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15801f4340 0x7f158014cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158612a640 1 -- 172.21.15.92:0/1221559200 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15801f4340 msgr2=0x7f158014cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158612a640 1 --2- 172.21.15.92:0/1221559200 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15801f4340 0x7f158014cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158612a640 1 -- 172.21.15.92:0/1221559200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f158014d450 msgr2=0x7f15801519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.873+0000 7f158612a640 1 --2- 172.21.15.92:0/1221559200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f158014d450 0x7f15801519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.874+0000 7f158612a640 1 -- 172.21.15.92:0/1221559200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1580070230 con 0x7f156c004800 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.874+0000 7f158612a640 1 --2- 172.21.15.92:0/1221559200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c004800 0x7f1580159b30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f157c00f880 tx=0x7f157c00fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.874+0000 7f1576ffd640 1 -- 172.21.15.92:0/1221559200 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c0050d0 con 0x7f156c004800 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.874+0000 7f1576ffd640 1 -- 172.21.15.92:0/1221559200 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f157c018070 con 0x7f156c004800 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.874+0000 7f158838e640 1 -- 172.21.15.92:0/1221559200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1580070510 con 0x7f156c004800 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.874+0000 7f1576ffd640 1 -- 172.21.15.92:0/1221559200 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c00d6d0 con 0x7f156c004800 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.874+0000 7f158838e640 1 -- 172.21.15.92:0/1221559200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1580158c20 con 0x7f156c004800 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.875+0000 7f1576ffd640 1 -- 172.21.15.92:0/1221559200 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f157c00d870 con 0x7f156c004800 2024-04-22T07:05:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.876+0000 7f1576ffd640 1 --2- 172.21.15.92:0/1221559200 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f156406c8f0 0x7f156406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.876+0000 7f1585929640 1 --2- 172.21.15.92:0/1221559200 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f156406c8f0 0x7f156406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.876+0000 7f1576ffd640 1 -- 172.21.15.92:0/1221559200 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f157c08b8b0 con 0x7f156c004800 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.876+0000 7f1585929640 1 --2- 172.21.15.92:0/1221559200 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f156406c8f0 0x7f156406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1570036000 tx=0x7f1570030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.877+0000 7f158af77740 1 Processor -- start 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.877+0000 7f158af77740 1 -- start start 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.877+0000 7f158af77740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 0x559220dfebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.877+0000 7f158af77740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 0x559220dff900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.877+0000 7f158af77740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220dbd810 0x559220dffe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.877+0000 7f158af77740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559220d24500 con 0x559220dfe7d0 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.877+0000 7f158af77740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559220cca6a0 con 0x559220de82d0 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.877+0000 7f158af77740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x559220d204c0 con 0x559220dbd810 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f158612a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 0x559220dfebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f158612a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 0x559220dfebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38868/0 (socket says 172.21.15.92:38868) 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f158612a640 1 -- 172.21.15.92:0/3042481687 learned_addr learned my addr 172.21.15.92:0/3042481687 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f158692b640 1 --2- 172.21.15.92:0/3042481687 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220dbd810 0x559220dffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f1585929640 1 --2- 172.21.15.92:0/3042481687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 0x559220dff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f158612a640 1 -- 172.21.15.92:0/3042481687 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220dbd810 msgr2=0x559220dffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f158612a640 1 --2- 172.21.15.92:0/3042481687 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220dbd810 0x559220dffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f158612a640 1 -- 172.21.15.92:0/3042481687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 msgr2=0x559220dff900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f158612a640 1 --2- 172.21.15.92:0/3042481687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 0x559220dff900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f158612a640 1 -- 172.21.15.92:0/3042481687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559220ce95e0 con 0x559220dfe7d0 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f158612a640 1 --2- 172.21.15.92:0/3042481687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 0x559220dfebb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f157c0050a0 tx=0x7f157c059010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2dbf62b82a0fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f1563fff640 1 -- 172.21.15.92:0/3042481687 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c0594d0 con 0x559220dfe7d0 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f1563fff640 1 -- 172.21.15.92:0/3042481687 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f157c08b050 con 0x559220dfe7d0 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.878+0000 7f1563fff640 1 -- 172.21.15.92:0/3042481687 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c05d880 con 0x559220dfe7d0 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.879+0000 7f158af77740 1 -- 172.21.15.92:0/3042481687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 msgr2=0x559220dfebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.879+0000 7f158af77740 1 --2- 172.21.15.92:0/3042481687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 0x559220dfebb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f157c0050a0 tx=0x7f157c059010 comp rx=0 tx=0).stop 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.879+0000 7f158af77740 1 -- 172.21.15.92:0/3042481687 shutdown_connections 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.879+0000 7f158af77740 1 --2- 172.21.15.92:0/3042481687 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220dbd810 0x559220dffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.879+0000 7f158af77740 1 --2- 172.21.15.92:0/3042481687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 0x559220dff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.879+0000 7f158af77740 1 --2- 172.21.15.92:0/3042481687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 0x559220dfebb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.879+0000 7f158af77740 1 -- 172.21.15.92:0/3042481687 >> 172.21.15.92:0/3042481687 conn(0x559220dbb550 msgr2=0x559220dbde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.879+0000 7f158af77740 1 -- 172.21.15.92:0/3042481687 shutdown_connections 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.879+0000 7f158af77740 1 -- 172.21.15.92:0/3042481687 wait complete. 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.879+0000 7f158af77740 1 -- start start 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.880+0000 7f158af77740 1 Processor -- start 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.880+0000 7f158af77740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 0x559220e025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.880+0000 7f158af77740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220dfe7d0 0x559220dbee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.880+0000 7f158af77740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220e02ba0 0x559220dd9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.880+0000 7f158af77740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559220dbcc30 con 0x559220e02ba0 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.880+0000 7f158af77740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559220cdda60 con 0x559220de82d0 2024-04-22T07:05:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.880+0000 7f158af77740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x559220de87a0 con 0x559220dfe7d0 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.880+0000 7f158af77740 10 client.0 _ll_register_callbacks cb 0x559220a36110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.880+0000 7f158692b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220e02ba0 0x559220dd9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f158692b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220e02ba0 0x559220dd9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38880/0 (socket says 172.21.15.92:38880) 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f158692b640 1 -- 172.21.15.92:0/3961014925 learned_addr learned my addr 172.21.15.92:0/3961014925 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f1585929640 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220dfe7d0 0x559220dbee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f158612a640 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 0x559220e025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f158692b640 1 -- 172.21.15.92:0/3961014925 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220dfe7d0 msgr2=0x559220dbee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f158692b640 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220dfe7d0 0x559220dbee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f158692b640 1 -- 172.21.15.92:0/3961014925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 msgr2=0x559220e025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f158692b640 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 0x559220e025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f158692b640 1 -- 172.21.15.92:0/3961014925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559220a36920 con 0x559220e02ba0 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f158692b640 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220e02ba0 0x559220dd9380 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f15800c4810 tx=0x7f1578007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57390e6f690cead server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f15617fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f15617fa640 1 -- 172.21.15.92:0/3961014925 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578004d70 con 0x559220e02ba0 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f15617fa640 1 -- 172.21.15.92:0/3961014925 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1578013830 con 0x559220e02ba0 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f158af77740 1 -- 172.21.15.92:0/3961014925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f157c08b050 con 0x559220e02ba0 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f15617fa640 1 -- 172.21.15.92:0/3961014925 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f1578025050 con 0x559220e02ba0 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f158af77740 10 client.5085 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f158af77740 1 -- 172.21.15.92:0/3961014925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559220b02d30 con 0x559220e02ba0 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.881+0000 7f15617fa640 1 -- 172.21.15.92:0/3961014925 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f157c08b050 con 0x559220e02ba0 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f15617fa640 1 -- 172.21.15.92:0/3961014925 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f1578028020 con 0x559220e02ba0 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f158af77740 10 client.5085 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f158af77740 10 client.5085 resolve_mds: validated gid 4275 aka daemon mds.e 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f158af77740 20 client.5085 populate_metadata read hostname 'smithi092' 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f158af77740 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x559220e131b0 0x559220e135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f158af77740 4 client.5085 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:05:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f158af77740 1 -- 172.21.15.92:0/3961014925 --> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559220cf7a10 con 0x559220e131b0 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f158612a640 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x559220e131b0 0x559220e135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f15617fa640 10 client.5085 ms_handle_connect on v2:172.21.15.92:6838/3493022212 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f158612a640 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x559220e131b0 0x559220e135c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6387515903bfbd10 server_cookie=151def746c31b67 in_seq=0 out_seq=0 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f15617fa640 1 -- 172.21.15.92:0/3961014925 <== mds.0 v2:172.21.15.92:6838/3493022212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559220cf7a10 con 0x559220e131b0 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f15617fa640 10 client.5085 handle_command_reply: tid=0 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.882+0000 7f158af77740 1 client.5085 shutdown 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.883+0000 7f158af77740 1 -- 172.21.15.92:0/3961014925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220e02ba0 msgr2=0x559220dd9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.883+0000 7f158af77740 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220e02ba0 0x559220dd9380 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f15800c4810 tx=0x7f1578007af0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.883+0000 7f158af77740 1 -- 172.21.15.92:0/3961014925 shutdown_connections 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.883+0000 7f158af77740 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x559220e131b0 0x559220e135c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.883+0000 7f158af77740 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220e02ba0 0x559220dd9380 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.883+0000 7f15617fa640 0 client.5085 ms_handle_reset on v2:172.21.15.92:6838/3493022212 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.883+0000 7f158af77740 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220dfe7d0 0x559220dbee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.883+0000 7f158af77740 1 --2- 172.21.15.92:0/3961014925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 0x559220e025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.883+0000 7f158af77740 1 -- 172.21.15.92:0/3961014925 >> 172.21.15.92:0/3961014925 conn(0x559220dbd810 msgr2=0x559220dbde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:21.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.883+0000 7f158af77740 1 -- 172.21.15.92:0/3961014925 shutdown_connections 2024-04-22T07:05:21.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.883+0000 7f158af77740 1 -- 172.21.15.92:0/3961014925 wait complete. 2024-04-22T07:05:21.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.883+0000 7f158af77740 20 client.5085 trim_cache size 0 max 16384 2024-04-22T07:05:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.900+0000 7f158af77740 1 Processor -- start 2024-04-22T07:05:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.900+0000 7f158af77740 1 -- start start 2024-04-22T07:05:21.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f158af77740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 0x559220dbdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f158af77740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 0x559220dbe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f158af77740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220e02ba0 0x559220e23820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f158af77740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f15801fe870 con 0x559220dfe7d0 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f158af77740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f15801fcd30 con 0x559220de82d0 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f158af77740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f15801fe1a0 con 0x559220e02ba0 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f1585929640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 0x559220dbe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f1585929640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 0x559220dbe130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38894/0 (socket says 172.21.15.92:38894) 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f1585929640 1 -- 172.21.15.92:0/2769997370 learned_addr learned my addr 172.21.15.92:0/2769997370 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f158692b640 1 --2- 172.21.15.92:0/2769997370 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220e02ba0 0x559220e23820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f1585929640 1 -- 172.21.15.92:0/2769997370 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220e02ba0 msgr2=0x559220e23820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f1585929640 1 --2- 172.21.15.92:0/2769997370 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220e02ba0 0x559220e23820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f158612a640 1 --2- 172.21.15.92:0/2769997370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 0x559220dbdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f1585929640 1 -- 172.21.15.92:0/2769997370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 msgr2=0x559220dbdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f1585929640 1 --2- 172.21.15.92:0/2769997370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 0x559220dbdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f1585929640 1 -- 172.21.15.92:0/2769997370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559220dfd580 con 0x559220dfe7d0 2024-04-22T07:05:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f158692b640 1 --2- 172.21.15.92:0/2769997370 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220e02ba0 0x559220e23820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f1585929640 1 --2- 172.21.15.92:0/2769997370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 0x559220dbe130 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1570004160 tx=0x7f157002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c2b791fa3a74c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f15637fe640 1 -- 172.21.15.92:0/2769997370 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570043040 con 0x559220dfe7d0 2024-04-22T07:05:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f15637fe640 1 -- 172.21.15.92:0/2769997370 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f157003a040 con 0x559220dfe7d0 2024-04-22T07:05:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.901+0000 7f15637fe640 1 -- 172.21.15.92:0/2769997370 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570039050 con 0x559220dfe7d0 2024-04-22T07:05:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.902+0000 7f158af77740 1 -- 172.21.15.92:0/2769997370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 msgr2=0x559220dbe130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.902+0000 7f158af77740 1 --2- 172.21.15.92:0/2769997370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 0x559220dbe130 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1570004160 tx=0x7f157002e000 comp rx=0 tx=0).stop 2024-04-22T07:05:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.902+0000 7f158af77740 1 -- 172.21.15.92:0/2769997370 shutdown_connections 2024-04-22T07:05:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.902+0000 7f158af77740 1 --2- 172.21.15.92:0/2769997370 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220e02ba0 0x559220e23820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.902+0000 7f158af77740 1 --2- 172.21.15.92:0/2769997370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220dfe7d0 0x559220dbe130 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.902+0000 7f158af77740 1 --2- 172.21.15.92:0/2769997370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220de82d0 0x559220dbdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.902+0000 7f158af77740 1 -- 172.21.15.92:0/2769997370 >> 172.21.15.92:0/2769997370 conn(0x559220dbd810 msgr2=0x559220db46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:21.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.902+0000 7f158af77740 1 -- 172.21.15.92:0/2769997370 shutdown_connections 2024-04-22T07:05:21.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.902+0000 7f158af77740 1 -- 172.21.15.92:0/2769997370 wait complete. 2024-04-22T07:05:21.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.902+0000 7f158af77740 1 -- start start 2024-04-22T07:05:21.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.903+0000 7f158af77740 1 Processor -- start 2024-04-22T07:05:21.904 INFO:tasks.ceph.mds.b.smithi145.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fc6cc0ba640 time 2024-04-22T07:05:21.906647+0000 2024-04-22T07:05:21.904 INFO:tasks.ceph.mds.b.smithi145.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.905 INFO:tasks.ceph.mds.f.smithi145.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fbb6cbf5640 time 2024-04-22T07:05:21.906671+0000 2024-04-22T07:05:21.905 INFO:tasks.ceph.mds.f.smithi145.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.905 INFO:tasks.ceph.mds.b.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.906 INFO:tasks.ceph.mds.b.smithi145.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fc6d9d81f5c] 2024-04-22T07:05:21.906 INFO:tasks.ceph.mds.b.smithi145.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc6d9d8211a] 2024-04-22T07:05:21.906 INFO:tasks.ceph.mds.b.smithi145.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56167734ffd2] 2024-04-22T07:05:21.906 INFO:tasks.ceph.mds.b.smithi145.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x561677350091] 2024-04-22T07:05:21.906 INFO:tasks.ceph.mds.b.smithi145.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56167730aa96] 2024-04-22T07:05:21.906 INFO:tasks.ceph.mds.b.smithi145.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56167730a9f6] 2024-04-22T07:05:21.906 INFO:tasks.ceph.mds.b.smithi145.stderr: 7: ceph-mds(+0x48e3cf) [0x5616774a83cf] 2024-04-22T07:05:21.906 INFO:tasks.ceph.mds.b.smithi145.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x5616774f8c1b] 2024-04-22T07:05:21.906 INFO:tasks.ceph.mds.b.smithi145.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fc6d9489c02] 2024-04-22T07:05:21.906 INFO:tasks.ceph.mds.b.smithi145.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fc6d950ec40] 2024-04-22T07:05:21.906 INFO:tasks.ceph.mds.b.smithi145.stderr:*** Caught signal (Aborted) ** 2024-04-22T07:05:21.906 INFO:tasks.ceph.mds.b.smithi145.stderr: in thread 7fc6cc0ba640 thread_name:quiesce.agt 2024-04-22T07:05:21.907 INFO:tasks.ceph.mds.f.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.907 INFO:tasks.ceph.mds.f.smithi145.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fbb7a781f5c] 2024-04-22T07:05:21.907 INFO:tasks.ceph.mds.f.smithi145.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fbb7a78211a] 2024-04-22T07:05:21.907 INFO:tasks.ceph.mds.f.smithi145.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a5d98efd2] 2024-04-22T07:05:21.907 INFO:tasks.ceph.mds.f.smithi145.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a5d98f091] 2024-04-22T07:05:21.907 INFO:tasks.ceph.mds.f.smithi145.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a5d949a96] 2024-04-22T07:05:21.907 INFO:tasks.ceph.mds.f.smithi145.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a5d9499f6] 2024-04-22T07:05:21.907 INFO:tasks.ceph.mds.f.smithi145.stderr: 7: ceph-mds(+0x48e3cf) [0x555a5dae73cf] 2024-04-22T07:05:21.907 INFO:tasks.ceph.mds.f.smithi145.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a5db37c1b] 2024-04-22T07:05:21.907 INFO:tasks.ceph.mds.f.smithi145.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fbb79e89c02] 2024-04-22T07:05:21.908 INFO:tasks.ceph.mds.f.smithi145.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fbb79f0ec40] 2024-04-22T07:05:21.908 INFO:tasks.ceph.mds.b.smithi145.stderr:2024-04-22T07:05:21.906+0000 7fc6cc0ba640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fc6cc0ba640 time 2024-04-22T07:05:21.906647+0000 2024-04-22T07:05:21.908 INFO:tasks.ceph.mds.b.smithi145.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.908 INFO:tasks.ceph.mds.b.smithi145.stderr: 2024-04-22T07:05:21.908 INFO:tasks.ceph.mds.b.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.909 INFO:tasks.ceph.mds.b.smithi145.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fc6d9d81f5c] 2024-04-22T07:05:21.909 INFO:tasks.ceph.mds.b.smithi145.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc6d9d8211a] 2024-04-22T07:05:21.909 INFO:tasks.ceph.mds.b.smithi145.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56167734ffd2] 2024-04-22T07:05:21.909 INFO:tasks.ceph.mds.b.smithi145.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x561677350091] 2024-04-22T07:05:21.909 INFO:tasks.ceph.mds.b.smithi145.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56167730aa96] 2024-04-22T07:05:21.909 INFO:tasks.ceph.mds.b.smithi145.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56167730a9f6] 2024-04-22T07:05:21.909 INFO:tasks.ceph.mds.b.smithi145.stderr: 7: ceph-mds(+0x48e3cf) [0x5616774a83cf] 2024-04-22T07:05:21.909 INFO:tasks.ceph.mds.b.smithi145.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x5616774f8c1b] 2024-04-22T07:05:21.909 INFO:tasks.ceph.mds.b.smithi145.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fc6d9489c02] 2024-04-22T07:05:21.909 INFO:tasks.ceph.mds.b.smithi145.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fc6d950ec40] 2024-04-22T07:05:21.909 INFO:tasks.ceph.mds.b.smithi145.stderr: 2024-04-22T07:05:21.910 INFO:tasks.ceph.mds.f.smithi145.stderr:*** Caught signal (Aborted) ** 2024-04-22T07:05:21.910 INFO:tasks.ceph.mds.f.smithi145.stderr: in thread 7fbb6cbf5640 thread_name:quiesce.agt 2024-04-22T07:05:21.910 INFO:tasks.ceph.mds.f.smithi145.stderr:2024-04-22T07:05:21.906+0000 7fbb6cbf5640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fbb6cbf5640 time 2024-04-22T07:05:21.906671+0000 2024-04-22T07:05:21.910 INFO:tasks.ceph.mds.f.smithi145.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.910 INFO:tasks.ceph.mds.f.smithi145.stderr: 2024-04-22T07:05:21.910 INFO:tasks.ceph.mds.f.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.910 INFO:tasks.ceph.mds.f.smithi145.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fbb7a781f5c] 2024-04-22T07:05:21.910 INFO:tasks.ceph.mds.f.smithi145.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fbb7a78211a] 2024-04-22T07:05:21.910 INFO:tasks.ceph.mds.f.smithi145.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a5d98efd2] 2024-04-22T07:05:21.910 INFO:tasks.ceph.mds.f.smithi145.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a5d98f091] 2024-04-22T07:05:21.910 INFO:tasks.ceph.mds.f.smithi145.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a5d949a96] 2024-04-22T07:05:21.910 INFO:tasks.ceph.mds.f.smithi145.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a5d9499f6] 2024-04-22T07:05:21.911 INFO:tasks.ceph.mds.f.smithi145.stderr: 7: ceph-mds(+0x48e3cf) [0x555a5dae73cf] 2024-04-22T07:05:21.911 INFO:tasks.ceph.mds.f.smithi145.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a5db37c1b] 2024-04-22T07:05:21.911 INFO:tasks.ceph.mds.f.smithi145.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fbb79e89c02] 2024-04-22T07:05:21.911 INFO:tasks.ceph.mds.f.smithi145.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fbb79f0ec40] 2024-04-22T07:05:21.911 INFO:tasks.ceph.mds.f.smithi145.stderr: 2024-04-22T07:05:21.911 INFO:tasks.ceph.mds.b.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.912 INFO:tasks.ceph.mds.b.smithi145.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fc6d943e6f0] 2024-04-22T07:05:21.912 INFO:tasks.ceph.mds.b.smithi145.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fc6d948b94c] 2024-04-22T07:05:21.912 INFO:tasks.ceph.mds.b.smithi145.stderr: 3: raise() 2024-04-22T07:05:21.912 INFO:tasks.ceph.mds.b.smithi145.stderr: 4: abort() 2024-04-22T07:05:21.912 INFO:tasks.ceph.mds.b.smithi145.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fc6d9d81fb6] 2024-04-22T07:05:21.912 INFO:tasks.ceph.mds.b.smithi145.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc6d9d8211a] 2024-04-22T07:05:21.912 INFO:tasks.ceph.mds.b.smithi145.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56167734ffd2] 2024-04-22T07:05:21.912 INFO:tasks.ceph.mds.b.smithi145.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x561677350091] 2024-04-22T07:05:21.912 INFO:tasks.ceph.mds.b.smithi145.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56167730aa96] 2024-04-22T07:05:21.912 INFO:tasks.ceph.mds.b.smithi145.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56167730a9f6] 2024-04-22T07:05:21.913 INFO:tasks.ceph.mds.b.smithi145.stderr: 11: ceph-mds(+0x48e3cf) [0x5616774a83cf] 2024-04-22T07:05:21.913 INFO:tasks.ceph.mds.b.smithi145.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x5616774f8c1b] 2024-04-22T07:05:21.913 INFO:tasks.ceph.mds.b.smithi145.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fc6d9489c02] 2024-04-22T07:05:21.913 INFO:tasks.ceph.mds.b.smithi145.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fc6d950ec40] 2024-04-22T07:05:21.913 INFO:tasks.ceph.mds.f.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.913 INFO:tasks.ceph.mds.f.smithi145.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbb79e3e6f0] 2024-04-22T07:05:21.913 INFO:tasks.ceph.mds.f.smithi145.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fbb79e8b94c] 2024-04-22T07:05:21.913 INFO:tasks.ceph.mds.f.smithi145.stderr: 3: raise() 2024-04-22T07:05:21.913 INFO:tasks.ceph.mds.f.smithi145.stderr: 4: abort() 2024-04-22T07:05:21.913 INFO:tasks.ceph.mds.f.smithi145.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fbb7a781fb6] 2024-04-22T07:05:21.914 INFO:tasks.ceph.mds.f.smithi145.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fbb7a78211a] 2024-04-22T07:05:21.914 INFO:tasks.ceph.mds.f.smithi145.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a5d98efd2] 2024-04-22T07:05:21.914 INFO:tasks.ceph.mds.f.smithi145.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a5d98f091] 2024-04-22T07:05:21.914 INFO:tasks.ceph.mds.f.smithi145.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a5d949a96] 2024-04-22T07:05:21.914 INFO:tasks.ceph.mds.f.smithi145.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a5d9499f6] 2024-04-22T07:05:21.914 INFO:tasks.ceph.mds.f.smithi145.stderr: 11: ceph-mds(+0x48e3cf) [0x555a5dae73cf] 2024-04-22T07:05:21.914 INFO:tasks.ceph.mds.f.smithi145.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a5db37c1b] 2024-04-22T07:05:21.914 INFO:tasks.ceph.mds.f.smithi145.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fbb79e89c02] 2024-04-22T07:05:21.914 INFO:tasks.ceph.mds.f.smithi145.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fbb79f0ec40] 2024-04-22T07:05:21.915 INFO:tasks.ceph.mds.b.smithi145.stderr:2024-04-22T07:05:21.907+0000 7fc6cc0ba640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:21.915 INFO:tasks.ceph.mds.b.smithi145.stderr: in thread 7fc6cc0ba640 thread_name:quiesce.agt 2024-04-22T07:05:21.915 INFO:tasks.ceph.mds.b.smithi145.stderr: 2024-04-22T07:05:21.915 INFO:tasks.ceph.mds.b.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.915 INFO:tasks.ceph.mds.b.smithi145.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fc6d943e6f0] 2024-04-22T07:05:21.915 INFO:tasks.ceph.mds.b.smithi145.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fc6d948b94c] 2024-04-22T07:05:21.915 INFO:tasks.ceph.mds.b.smithi145.stderr: 3: raise() 2024-04-22T07:05:21.915 INFO:tasks.ceph.mds.b.smithi145.stderr: 4: abort() 2024-04-22T07:05:21.915 INFO:tasks.ceph.mds.b.smithi145.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fc6d9d81fb6] 2024-04-22T07:05:21.916 INFO:tasks.ceph.mds.b.smithi145.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc6d9d8211a] 2024-04-22T07:05:21.916 INFO:tasks.ceph.mds.b.smithi145.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56167734ffd2] 2024-04-22T07:05:21.916 INFO:tasks.ceph.mds.b.smithi145.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x561677350091] 2024-04-22T07:05:21.916 INFO:tasks.ceph.mds.b.smithi145.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56167730aa96] 2024-04-22T07:05:21.916 INFO:tasks.ceph.mds.b.smithi145.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56167730a9f6] 2024-04-22T07:05:21.916 INFO:tasks.ceph.mds.b.smithi145.stderr: 11: ceph-mds(+0x48e3cf) [0x5616774a83cf] 2024-04-22T07:05:21.916 INFO:tasks.ceph.mds.b.smithi145.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x5616774f8c1b] 2024-04-22T07:05:21.916 INFO:tasks.ceph.mds.b.smithi145.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fc6d9489c02] 2024-04-22T07:05:21.916 INFO:tasks.ceph.mds.b.smithi145.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fc6d950ec40] 2024-04-22T07:05:21.916 INFO:tasks.ceph.mds.b.smithi145.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:21.916 INFO:tasks.ceph.mds.b.smithi145.stderr: 2024-04-22T07:05:21.917 INFO:tasks.ceph.mds.f.smithi145.stderr:2024-04-22T07:05:21.907+0000 7fbb6cbf5640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:21.917 INFO:tasks.ceph.mds.f.smithi145.stderr: in thread 7fbb6cbf5640 thread_name:quiesce.agt 2024-04-22T07:05:21.917 INFO:tasks.ceph.mds.f.smithi145.stderr: 2024-04-22T07:05:21.917 INFO:tasks.ceph.mds.f.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.917 INFO:tasks.ceph.mds.f.smithi145.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbb79e3e6f0] 2024-04-22T07:05:21.917 INFO:tasks.ceph.mds.f.smithi145.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fbb79e8b94c] 2024-04-22T07:05:21.917 INFO:tasks.ceph.mds.f.smithi145.stderr: 3: raise() 2024-04-22T07:05:21.917 INFO:tasks.ceph.mds.f.smithi145.stderr: 4: abort() 2024-04-22T07:05:21.917 INFO:tasks.ceph.mds.f.smithi145.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fbb7a781fb6] 2024-04-22T07:05:21.917 INFO:tasks.ceph.mds.f.smithi145.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fbb7a78211a] 2024-04-22T07:05:21.917 INFO:tasks.ceph.mds.f.smithi145.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a5d98efd2] 2024-04-22T07:05:21.918 INFO:tasks.ceph.mds.f.smithi145.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a5d98f091] 2024-04-22T07:05:21.918 INFO:tasks.ceph.mds.f.smithi145.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a5d949a96] 2024-04-22T07:05:21.918 INFO:tasks.ceph.mds.f.smithi145.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a5d9499f6] 2024-04-22T07:05:21.918 INFO:tasks.ceph.mds.f.smithi145.stderr: 11: ceph-mds(+0x48e3cf) [0x555a5dae73cf] 2024-04-22T07:05:21.918 INFO:tasks.ceph.mds.f.smithi145.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a5db37c1b] 2024-04-22T07:05:21.918 INFO:tasks.ceph.mds.f.smithi145.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fbb79e89c02] 2024-04-22T07:05:21.918 INFO:tasks.ceph.mds.f.smithi145.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fbb79f0ec40] 2024-04-22T07:05:21.918 INFO:tasks.ceph.mds.f.smithi145.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:21.918 INFO:tasks.ceph.mds.f.smithi145.stderr: 2024-04-22T07:05:21.919 INFO:tasks.ceph.mds.b.smithi145.stderr: -1> 2024-04-22T07:05:21.906+0000 7fc6cc0ba640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fc6cc0ba640 time 2024-04-22T07:05:21.906647+0000 2024-04-22T07:05:21.919 INFO:tasks.ceph.mds.b.smithi145.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.919 INFO:tasks.ceph.mds.b.smithi145.stderr: 2024-04-22T07:05:21.919 INFO:tasks.ceph.mds.b.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.919 INFO:tasks.ceph.mds.b.smithi145.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fc6d9d81f5c] 2024-04-22T07:05:21.919 INFO:tasks.ceph.mds.b.smithi145.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc6d9d8211a] 2024-04-22T07:05:21.919 INFO:tasks.ceph.mds.b.smithi145.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56167734ffd2] 2024-04-22T07:05:21.919 INFO:tasks.ceph.mds.b.smithi145.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x561677350091] 2024-04-22T07:05:21.919 INFO:tasks.ceph.mds.b.smithi145.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56167730aa96] 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56167730a9f6] 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 7: ceph-mds(+0x48e3cf) [0x5616774a83cf] 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x5616774f8c1b] 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fc6d9489c02] 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fc6d950ec40] 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 0> 2024-04-22T07:05:21.907+0000 7fc6cc0ba640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: in thread 7fc6cc0ba640 thread_name:quiesce.agt 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fc6d943e6f0] 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fc6d948b94c] 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 3: raise() 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 4: abort() 2024-04-22T07:05:21.920 INFO:tasks.ceph.mds.b.smithi145.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fc6d9d81fb6] 2024-04-22T07:05:21.921 INFO:tasks.ceph.mds.b.smithi145.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc6d9d8211a] 2024-04-22T07:05:21.921 INFO:tasks.ceph.mds.b.smithi145.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56167734ffd2] 2024-04-22T07:05:21.921 INFO:tasks.ceph.mds.b.smithi145.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x561677350091] 2024-04-22T07:05:21.921 INFO:tasks.ceph.mds.b.smithi145.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56167730aa96] 2024-04-22T07:05:21.921 INFO:tasks.ceph.mds.b.smithi145.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56167730a9f6] 2024-04-22T07:05:21.921 INFO:tasks.ceph.mds.b.smithi145.stderr: 11: ceph-mds(+0x48e3cf) [0x5616774a83cf] 2024-04-22T07:05:21.921 INFO:tasks.ceph.mds.b.smithi145.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x5616774f8c1b] 2024-04-22T07:05:21.921 INFO:tasks.ceph.mds.b.smithi145.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fc6d9489c02] 2024-04-22T07:05:21.921 INFO:tasks.ceph.mds.b.smithi145.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fc6d950ec40] 2024-04-22T07:05:21.921 INFO:tasks.ceph.mds.b.smithi145.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:21.921 INFO:tasks.ceph.mds.b.smithi145.stderr: 2024-04-22T07:05:21.922 INFO:tasks.ceph.mds.f.smithi145.stderr: -1> 2024-04-22T07:05:21.906+0000 7fbb6cbf5640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fbb6cbf5640 time 2024-04-22T07:05:21.906671+0000 2024-04-22T07:05:21.922 INFO:tasks.ceph.mds.f.smithi145.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.922 INFO:tasks.ceph.mds.f.smithi145.stderr: 2024-04-22T07:05:21.922 INFO:tasks.ceph.mds.f.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.922 INFO:tasks.ceph.mds.f.smithi145.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fbb7a781f5c] 2024-04-22T07:05:21.922 INFO:tasks.ceph.mds.f.smithi145.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fbb7a78211a] 2024-04-22T07:05:21.922 INFO:tasks.ceph.mds.f.smithi145.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a5d98efd2] 2024-04-22T07:05:21.922 INFO:tasks.ceph.mds.f.smithi145.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a5d98f091] 2024-04-22T07:05:21.922 INFO:tasks.ceph.mds.f.smithi145.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a5d949a96] 2024-04-22T07:05:21.922 INFO:tasks.ceph.mds.f.smithi145.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a5d9499f6] 2024-04-22T07:05:21.922 INFO:tasks.ceph.mds.f.smithi145.stderr: 7: ceph-mds(+0x48e3cf) [0x555a5dae73cf] 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a5db37c1b] 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fbb79e89c02] 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fbb79f0ec40] 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 0> 2024-04-22T07:05:21.907+0000 7fbb6cbf5640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: in thread 7fbb6cbf5640 thread_name:quiesce.agt 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbb79e3e6f0] 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fbb79e8b94c] 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 3: raise() 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 4: abort() 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fbb7a781fb6] 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fbb7a78211a] 2024-04-22T07:05:21.923 INFO:tasks.ceph.mds.f.smithi145.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a5d98efd2] 2024-04-22T07:05:21.924 INFO:tasks.ceph.mds.f.smithi145.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a5d98f091] 2024-04-22T07:05:21.924 INFO:tasks.ceph.mds.f.smithi145.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a5d949a96] 2024-04-22T07:05:21.924 INFO:tasks.ceph.mds.f.smithi145.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a5d9499f6] 2024-04-22T07:05:21.924 INFO:tasks.ceph.mds.f.smithi145.stderr: 11: ceph-mds(+0x48e3cf) [0x555a5dae73cf] 2024-04-22T07:05:21.924 INFO:tasks.ceph.mds.f.smithi145.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a5db37c1b] 2024-04-22T07:05:21.924 INFO:tasks.ceph.mds.f.smithi145.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fbb79e89c02] 2024-04-22T07:05:21.924 INFO:tasks.ceph.mds.f.smithi145.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fbb79f0ec40] 2024-04-22T07:05:21.924 INFO:tasks.ceph.mds.f.smithi145.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:21.924 INFO:tasks.ceph.mds.f.smithi145.stderr: 2024-04-22T07:05:21.925 INFO:tasks.ceph.mds.b.smithi145.stderr: -1> 2024-04-22T07:05:21.906+0000 7fc6cc0ba640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fc6cc0ba640 time 2024-04-22T07:05:21.906647+0000 2024-04-22T07:05:21.925 INFO:tasks.ceph.mds.b.smithi145.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.925 INFO:tasks.ceph.mds.b.smithi145.stderr: 2024-04-22T07:05:21.925 INFO:tasks.ceph.mds.b.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.925 INFO:tasks.ceph.mds.b.smithi145.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fc6d9d81f5c] 2024-04-22T07:05:21.925 INFO:tasks.ceph.mds.b.smithi145.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc6d9d8211a] 2024-04-22T07:05:21.925 INFO:tasks.ceph.mds.b.smithi145.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56167734ffd2] 2024-04-22T07:05:21.925 INFO:tasks.ceph.mds.b.smithi145.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x561677350091] 2024-04-22T07:05:21.925 INFO:tasks.ceph.mds.b.smithi145.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56167730aa96] 2024-04-22T07:05:21.925 INFO:tasks.ceph.mds.b.smithi145.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56167730a9f6] 2024-04-22T07:05:21.925 INFO:tasks.ceph.mds.b.smithi145.stderr: 7: ceph-mds(+0x48e3cf) [0x5616774a83cf] 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x5616774f8c1b] 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fc6d9489c02] 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fc6d950ec40] 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: 0> 2024-04-22T07:05:21.907+0000 7fc6cc0ba640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: in thread 7fc6cc0ba640 thread_name:quiesce.agt 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fc6d943e6f0] 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fc6d948b94c] 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: 3: raise() 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: 4: abort() 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fc6d9d81fb6] 2024-04-22T07:05:21.926 INFO:tasks.ceph.mds.b.smithi145.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc6d9d8211a] 2024-04-22T07:05:21.927 INFO:tasks.ceph.mds.b.smithi145.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56167734ffd2] 2024-04-22T07:05:21.927 INFO:tasks.ceph.mds.b.smithi145.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x561677350091] 2024-04-22T07:05:21.927 INFO:tasks.ceph.mds.b.smithi145.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56167730aa96] 2024-04-22T07:05:21.927 INFO:tasks.ceph.mds.b.smithi145.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56167730a9f6] 2024-04-22T07:05:21.927 INFO:tasks.ceph.mds.b.smithi145.stderr: 11: ceph-mds(+0x48e3cf) [0x5616774a83cf] 2024-04-22T07:05:21.927 INFO:tasks.ceph.mds.b.smithi145.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x5616774f8c1b] 2024-04-22T07:05:21.927 INFO:tasks.ceph.mds.b.smithi145.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fc6d9489c02] 2024-04-22T07:05:21.927 INFO:tasks.ceph.mds.b.smithi145.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fc6d950ec40] 2024-04-22T07:05:21.927 INFO:tasks.ceph.mds.b.smithi145.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:21.927 INFO:tasks.ceph.mds.b.smithi145.stderr: 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: -1> 2024-04-22T07:05:21.906+0000 7fbb6cbf5640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fbb6cbf5640 time 2024-04-22T07:05:21.906671+0000 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fbb7a781f5c] 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fbb7a78211a] 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a5d98efd2] 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a5d98f091] 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a5d949a96] 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a5d9499f6] 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: 7: ceph-mds(+0x48e3cf) [0x555a5dae73cf] 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a5db37c1b] 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fbb79e89c02] 2024-04-22T07:05:21.928 INFO:tasks.ceph.mds.f.smithi145.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fbb79f0ec40] 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 0> 2024-04-22T07:05:21.907+0000 7fbb6cbf5640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: in thread 7fbb6cbf5640 thread_name:quiesce.agt 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbb79e3e6f0] 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fbb79e8b94c] 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 3: raise() 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 4: abort() 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fbb7a781fb6] 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fbb7a78211a] 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a5d98efd2] 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a5d98f091] 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a5d949a96] 2024-04-22T07:05:21.929 INFO:tasks.ceph.mds.f.smithi145.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a5d9499f6] 2024-04-22T07:05:21.930 INFO:tasks.ceph.mds.f.smithi145.stderr: 11: ceph-mds(+0x48e3cf) [0x555a5dae73cf] 2024-04-22T07:05:21.930 INFO:tasks.ceph.mds.f.smithi145.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a5db37c1b] 2024-04-22T07:05:21.930 INFO:tasks.ceph.mds.f.smithi145.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fbb79e89c02] 2024-04-22T07:05:21.930 INFO:tasks.ceph.mds.f.smithi145.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fbb79f0ec40] 2024-04-22T07:05:21.930 INFO:tasks.ceph.mds.f.smithi145.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:21.930 INFO:tasks.ceph.mds.f.smithi145.stderr: 2024-04-22T07:05:21.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.903+0000 7f158af77740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220de82d0 0x559220dbdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.903+0000 7f158af77740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220dfe7d0 0x559220de5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.903+0000 7f158af77740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220e02ba0 0x559220de6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.903+0000 7f158af77740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f15801fdad0 con 0x559220de82d0 2024-04-22T07:05:21.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.903+0000 7f158af77740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f15801fd400 con 0x559220dfe7d0 2024-04-22T07:05:21.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.903+0000 7f158af77740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f15800c61f0 con 0x559220e02ba0 2024-04-22T07:05:21.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.903+0000 7f158af77740 10 client.0 _ll_register_callbacks cb 0x7f1578005c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.903+0000 7f158612a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220de82d0 0x559220dbdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.903+0000 7f158612a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220de82d0 0x559220dbdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38902/0 (socket says 172.21.15.92:38902) 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.903+0000 7f158612a640 1 -- 172.21.15.92:0/3708777330 learned_addr learned my addr 172.21.15.92:0/3708777330 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158692b640 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220e02ba0 0x559220de6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f1585929640 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220dfe7d0 0x559220de5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158612a640 1 -- 172.21.15.92:0/3708777330 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220e02ba0 msgr2=0x559220de6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158612a640 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220e02ba0 0x559220de6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158612a640 1 -- 172.21.15.92:0/3708777330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220dfe7d0 msgr2=0x559220de5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158612a640 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220dfe7d0 0x559220de5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158612a640 1 -- 172.21.15.92:0/3708777330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559220dbfae0 con 0x559220de82d0 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158692b640 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220e02ba0 0x559220de6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f1585929640 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220dfe7d0 0x559220de5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158612a640 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220de82d0 0x559220dbdff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f157c08bbd0 tx=0x7f157c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcef68b81fcd7dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:21.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f15617fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f15617fa640 1 -- 172.21.15.92:0/3708777330 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c0594d0 con 0x559220de82d0 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f15617fa640 1 -- 172.21.15.92:0/3708777330 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f157c0577d0 con 0x559220de82d0 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158af77740 1 -- 172.21.15.92:0/3708777330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f157003a040 con 0x559220de82d0 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f15617fa640 1 -- 172.21.15.92:0/3708777330 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f157c05d880 con 0x559220de82d0 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158af77740 10 client.5091 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158af77740 1 -- 172.21.15.92:0/3708777330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559220dbe770 con 0x559220de82d0 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f15617fa640 1 -- 172.21.15.92:0/3708777330 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f157003a040 con 0x559220de82d0 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f15617fa640 1 -- 172.21.15.92:0/3708777330 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f157c055b40 con 0x559220de82d0 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158af77740 10 client.5091 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158af77740 10 client.5091 resolve_mds: validated gid 4275 aka daemon mds.e 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158af77740 20 client.5091 populate_metadata read hostname 'smithi092' 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158af77740 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x559220e199e0 0x559220de6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158af77740 4 client.5091 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2853aa63", "release": true, "await": true}] 2024-04-22T07:05:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.904+0000 7f158af77740 1 -- 172.21.15.92:0/3708777330 --> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2853aa63", "release": true, "await": true}) -- 0x559220d31f20 con 0x559220e199e0 2024-04-22T07:05:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.905+0000 7f1585929640 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x559220e199e0 0x559220de6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.905+0000 7f15617fa640 10 client.5091 ms_handle_connect on v2:172.21.15.92:6838/3493022212 2024-04-22T07:05:21.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:21.905+0000 7f1585929640 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x559220e199e0 0x559220de6850 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ee13a6d2e1861e server_cookie=942015d53c74f3b3 in_seq=0 out_seq=0 2024-04-22T07:05:21.933 INFO:tasks.ceph.mds.c.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fd189d9a640 time 2024-04-22T07:05:21.906573+0000 2024-04-22T07:05:21.933 INFO:tasks.ceph.mds.c.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.933 INFO:tasks.ceph.mds.a.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f56a3c30640 time 2024-04-22T07:05:21.906600+0000 2024-04-22T07:05:21.933 INFO:tasks.ceph.mds.a.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.934 INFO:tasks.ceph.mds.c.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.934 INFO:tasks.ceph.mds.c.smithi092.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd197981f5c] 2024-04-22T07:05:21.934 INFO:tasks.ceph.mds.c.smithi092.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd19798211a] 2024-04-22T07:05:21.934 INFO:tasks.ceph.mds.c.smithi092.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ed9aa20fd2] 2024-04-22T07:05:21.934 INFO:tasks.ceph.mds.c.smithi092.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ed9aa21091] 2024-04-22T07:05:21.934 INFO:tasks.ceph.mds.c.smithi092.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ed9a9dba96] 2024-04-22T07:05:21.934 INFO:tasks.ceph.mds.c.smithi092.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ed9a9db9f6] 2024-04-22T07:05:21.934 INFO:tasks.ceph.mds.c.smithi092.stderr: 7: ceph-mds(+0x48e3cf) [0x55ed9ab793cf] 2024-04-22T07:05:21.934 INFO:tasks.ceph.mds.c.smithi092.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ed9abc9c1b] 2024-04-22T07:05:21.934 INFO:tasks.ceph.mds.c.smithi092.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fd197089c02] 2024-04-22T07:05:21.935 INFO:tasks.ceph.mds.c.smithi092.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fd19710ec40] 2024-04-22T07:05:21.935 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.935 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f56b1781f5c] 2024-04-22T07:05:21.935 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f56b178211a] 2024-04-22T07:05:21.935 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56425b97afd2] 2024-04-22T07:05:21.935 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x56425b97b091] 2024-04-22T07:05:21.935 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56425b935a96] 2024-04-22T07:05:21.935 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56425b9359f6] 2024-04-22T07:05:21.935 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: ceph-mds(+0x48e3cf) [0x56425bad33cf] 2024-04-22T07:05:21.935 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x56425bb23c1b] 2024-04-22T07:05:21.936 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f56b0e89c02] 2024-04-22T07:05:21.936 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f56b0f0ec40] 2024-04-22T07:05:21.936 INFO:tasks.ceph.mds.a.smithi092.stderr:2024-04-22T07:05:21.906+0000 7f56a3c30640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f56a3c30640 time 2024-04-22T07:05:21.906600+0000 2024-04-22T07:05:21.936 INFO:tasks.ceph.mds.a.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.936 INFO:tasks.ceph.mds.a.smithi092.stderr: 2024-04-22T07:05:21.936 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.936 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f56b1781f5c] 2024-04-22T07:05:21.936 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f56b178211a] 2024-04-22T07:05:21.936 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56425b97afd2] 2024-04-22T07:05:21.936 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x56425b97b091] 2024-04-22T07:05:21.936 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56425b935a96] 2024-04-22T07:05:21.936 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56425b9359f6] 2024-04-22T07:05:21.937 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: ceph-mds(+0x48e3cf) [0x56425bad33cf] 2024-04-22T07:05:21.937 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x56425bb23c1b] 2024-04-22T07:05:21.937 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f56b0e89c02] 2024-04-22T07:05:21.937 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f56b0f0ec40] 2024-04-22T07:05:21.937 INFO:tasks.ceph.mds.a.smithi092.stderr: 2024-04-22T07:05:21.937 INFO:tasks.ceph.mds.a.smithi092.stderr:*** Caught signal (Aborted) ** 2024-04-22T07:05:21.937 INFO:tasks.ceph.mds.a.smithi092.stderr: in thread 7f56a3c30640 thread_name:quiesce.agt 2024-04-22T07:05:21.937 INFO:tasks.ceph.mds.c.smithi092.stderr:2024-04-22T07:05:21.906+0000 7fd189d9a640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fd189d9a640 time 2024-04-22T07:05:21.906573+0000 2024-04-22T07:05:21.937 INFO:tasks.ceph.mds.c.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.937 INFO:tasks.ceph.mds.c.smithi092.stderr: 2024-04-22T07:05:21.937 INFO:tasks.ceph.mds.c.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.938 INFO:tasks.ceph.mds.c.smithi092.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd197981f5c] 2024-04-22T07:05:21.938 INFO:tasks.ceph.mds.c.smithi092.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd19798211a] 2024-04-22T07:05:21.938 INFO:tasks.ceph.mds.c.smithi092.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ed9aa20fd2] 2024-04-22T07:05:21.938 INFO:tasks.ceph.mds.c.smithi092.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ed9aa21091] 2024-04-22T07:05:21.938 INFO:tasks.ceph.mds.c.smithi092.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ed9a9dba96] 2024-04-22T07:05:21.938 INFO:tasks.ceph.mds.c.smithi092.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ed9a9db9f6] 2024-04-22T07:05:21.938 INFO:tasks.ceph.mds.c.smithi092.stderr: 7: ceph-mds(+0x48e3cf) [0x55ed9ab793cf] 2024-04-22T07:05:21.938 INFO:tasks.ceph.mds.c.smithi092.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ed9abc9c1b] 2024-04-22T07:05:21.938 INFO:tasks.ceph.mds.c.smithi092.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fd197089c02] 2024-04-22T07:05:21.938 INFO:tasks.ceph.mds.c.smithi092.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fd19710ec40] 2024-04-22T07:05:21.938 INFO:tasks.ceph.mds.c.smithi092.stderr: 2024-04-22T07:05:21.939 INFO:tasks.ceph.mds.c.smithi092.stderr:*** Caught signal (Aborted) ** 2024-04-22T07:05:21.939 INFO:tasks.ceph.mds.c.smithi092.stderr: in thread 7fd189d9a640 thread_name:quiesce.agt 2024-04-22T07:05:21.939 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.939 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f56b0e3e6f0] 2024-04-22T07:05:21.939 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f56b0e8b94c] 2024-04-22T07:05:21.939 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: raise() 2024-04-22T07:05:21.939 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: abort() 2024-04-22T07:05:21.939 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f56b1781fb6] 2024-04-22T07:05:21.939 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f56b178211a] 2024-04-22T07:05:21.939 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56425b97afd2] 2024-04-22T07:05:21.939 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x56425b97b091] 2024-04-22T07:05:21.940 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56425b935a96] 2024-04-22T07:05:21.940 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56425b9359f6] 2024-04-22T07:05:21.940 INFO:tasks.ceph.mds.a.smithi092.stderr: 11: ceph-mds(+0x48e3cf) [0x56425bad33cf] 2024-04-22T07:05:21.940 INFO:tasks.ceph.mds.a.smithi092.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x56425bb23c1b] 2024-04-22T07:05:21.940 INFO:tasks.ceph.mds.a.smithi092.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f56b0e89c02] 2024-04-22T07:05:21.940 INFO:tasks.ceph.mds.a.smithi092.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f56b0f0ec40] 2024-04-22T07:05:21.940 INFO:tasks.ceph.mds.c.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.940 INFO:tasks.ceph.mds.c.smithi092.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fd19703e6f0] 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fd19708b94c] 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 3: raise() 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 4: abort() 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd197981fb6] 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd19798211a] 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ed9aa20fd2] 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ed9aa21091] 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ed9a9dba96] 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ed9a9db9f6] 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 11: ceph-mds(+0x48e3cf) [0x55ed9ab793cf] 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ed9abc9c1b] 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fd197089c02] 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fd19710ec40] 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr:2024-04-22T07:05:21.907+0000 7fd189d9a640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:21.941 INFO:tasks.ceph.mds.c.smithi092.stderr: in thread 7fd189d9a640 thread_name:quiesce.agt 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fd19703e6f0] 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fd19708b94c] 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: 3: raise() 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: 4: abort() 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd197981fb6] 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd19798211a] 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ed9aa20fd2] 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ed9aa21091] 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ed9a9dba96] 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ed9a9db9f6] 2024-04-22T07:05:21.942 INFO:tasks.ceph.mds.c.smithi092.stderr: 11: ceph-mds(+0x48e3cf) [0x55ed9ab793cf] 2024-04-22T07:05:21.943 INFO:tasks.ceph.mds.c.smithi092.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ed9abc9c1b] 2024-04-22T07:05:21.943 INFO:tasks.ceph.mds.c.smithi092.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fd197089c02] 2024-04-22T07:05:21.943 INFO:tasks.ceph.mds.c.smithi092.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fd19710ec40] 2024-04-22T07:05:21.943 INFO:tasks.ceph.mds.c.smithi092.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:21.943 INFO:tasks.ceph.mds.c.smithi092.stderr: 2024-04-22T07:05:21.943 INFO:tasks.ceph.mds.a.smithi092.stderr:2024-04-22T07:05:21.907+0000 7f56a3c30640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:21.943 INFO:tasks.ceph.mds.a.smithi092.stderr: in thread 7f56a3c30640 thread_name:quiesce.agt 2024-04-22T07:05:21.943 INFO:tasks.ceph.mds.a.smithi092.stderr: 2024-04-22T07:05:21.943 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.943 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f56b0e3e6f0] 2024-04-22T07:05:21.943 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f56b0e8b94c] 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: raise() 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: abort() 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f56b1781fb6] 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f56b178211a] 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56425b97afd2] 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x56425b97b091] 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56425b935a96] 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56425b9359f6] 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 11: ceph-mds(+0x48e3cf) [0x56425bad33cf] 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x56425bb23c1b] 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f56b0e89c02] 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f56b0f0ec40] 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:21.944 INFO:tasks.ceph.mds.a.smithi092.stderr: 2024-04-22T07:05:21.945 INFO:tasks.ceph.mds.c.smithi092.stderr: -2> 2024-04-22T07:05:21.906+0000 7fd189d9a640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fd189d9a640 time 2024-04-22T07:05:21.906573+0000 2024-04-22T07:05:21.945 INFO:tasks.ceph.mds.c.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.945 INFO:tasks.ceph.mds.c.smithi092.stderr: 2024-04-22T07:05:21.945 INFO:tasks.ceph.mds.c.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.945 INFO:tasks.ceph.mds.c.smithi092.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd197981f5c] 2024-04-22T07:05:21.945 INFO:tasks.ceph.mds.c.smithi092.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd19798211a] 2024-04-22T07:05:21.945 INFO:tasks.ceph.mds.c.smithi092.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ed9aa20fd2] 2024-04-22T07:05:21.945 INFO:tasks.ceph.mds.c.smithi092.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ed9aa21091] 2024-04-22T07:05:21.945 INFO:tasks.ceph.mds.c.smithi092.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ed9a9dba96] 2024-04-22T07:05:21.945 INFO:tasks.ceph.mds.c.smithi092.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ed9a9db9f6] 2024-04-22T07:05:21.945 INFO:tasks.ceph.mds.c.smithi092.stderr: 7: ceph-mds(+0x48e3cf) [0x55ed9ab793cf] 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ed9abc9c1b] 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fd197089c02] 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fd19710ec40] 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: 0> 2024-04-22T07:05:21.907+0000 7fd189d9a640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: in thread 7fd189d9a640 thread_name:quiesce.agt 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fd19703e6f0] 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fd19708b94c] 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: 3: raise() 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: 4: abort() 2024-04-22T07:05:21.946 INFO:tasks.ceph.mds.c.smithi092.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd197981fb6] 2024-04-22T07:05:21.947 INFO:tasks.ceph.mds.c.smithi092.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd19798211a] 2024-04-22T07:05:21.947 INFO:tasks.ceph.mds.c.smithi092.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ed9aa20fd2] 2024-04-22T07:05:21.947 INFO:tasks.ceph.mds.c.smithi092.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ed9aa21091] 2024-04-22T07:05:21.947 INFO:tasks.ceph.mds.c.smithi092.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ed9a9dba96] 2024-04-22T07:05:21.947 INFO:tasks.ceph.mds.c.smithi092.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ed9a9db9f6] 2024-04-22T07:05:21.947 INFO:tasks.ceph.mds.c.smithi092.stderr: 11: ceph-mds(+0x48e3cf) [0x55ed9ab793cf] 2024-04-22T07:05:21.947 INFO:tasks.ceph.mds.c.smithi092.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ed9abc9c1b] 2024-04-22T07:05:21.947 INFO:tasks.ceph.mds.c.smithi092.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fd197089c02] 2024-04-22T07:05:21.947 INFO:tasks.ceph.mds.c.smithi092.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fd19710ec40] 2024-04-22T07:05:21.947 INFO:tasks.ceph.mds.c.smithi092.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:21.947 INFO:tasks.ceph.mds.c.smithi092.stderr: 2024-04-22T07:05:21.947 INFO:tasks.ceph.mds.a.smithi092.stderr: -1> 2024-04-22T07:05:21.906+0000 7f56a3c30640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f56a3c30640 time 2024-04-22T07:05:21.906600+0000 2024-04-22T07:05:21.948 INFO:tasks.ceph.mds.a.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.948 INFO:tasks.ceph.mds.a.smithi092.stderr: 2024-04-22T07:05:21.948 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.948 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f56b1781f5c] 2024-04-22T07:05:21.948 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f56b178211a] 2024-04-22T07:05:21.948 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56425b97afd2] 2024-04-22T07:05:21.948 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x56425b97b091] 2024-04-22T07:05:21.948 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56425b935a96] 2024-04-22T07:05:21.948 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56425b9359f6] 2024-04-22T07:05:21.948 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: ceph-mds(+0x48e3cf) [0x56425bad33cf] 2024-04-22T07:05:21.948 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x56425bb23c1b] 2024-04-22T07:05:21.948 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f56b0e89c02] 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f56b0f0ec40] 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 0> 2024-04-22T07:05:21.907+0000 7f56a3c30640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: in thread 7f56a3c30640 thread_name:quiesce.agt 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f56b0e3e6f0] 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f56b0e8b94c] 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: raise() 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: abort() 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f56b1781fb6] 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f56b178211a] 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56425b97afd2] 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x56425b97b091] 2024-04-22T07:05:21.949 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56425b935a96] 2024-04-22T07:05:21.950 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56425b9359f6] 2024-04-22T07:05:21.950 INFO:tasks.ceph.mds.a.smithi092.stderr: 11: ceph-mds(+0x48e3cf) [0x56425bad33cf] 2024-04-22T07:05:21.950 INFO:tasks.ceph.mds.a.smithi092.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x56425bb23c1b] 2024-04-22T07:05:21.950 INFO:tasks.ceph.mds.a.smithi092.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f56b0e89c02] 2024-04-22T07:05:21.950 INFO:tasks.ceph.mds.a.smithi092.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f56b0f0ec40] 2024-04-22T07:05:21.950 INFO:tasks.ceph.mds.a.smithi092.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:21.950 INFO:tasks.ceph.mds.a.smithi092.stderr: 2024-04-22T07:05:21.950 INFO:tasks.ceph.mds.c.smithi092.stderr: -5> 2024-04-22T07:05:21.906+0000 7fd189d9a640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fd189d9a640 time 2024-04-22T07:05:21.906573+0000 2024-04-22T07:05:21.950 INFO:tasks.ceph.mds.c.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd197981f5c] 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd19798211a] 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ed9aa20fd2] 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ed9aa21091] 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ed9a9dba96] 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ed9a9db9f6] 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: 7: ceph-mds(+0x48e3cf) [0x55ed9ab793cf] 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ed9abc9c1b] 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fd197089c02] 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fd19710ec40] 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: -3> 2024-04-22T07:05:21.907+0000 7fd189d9a640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:21.951 INFO:tasks.ceph.mds.c.smithi092.stderr: in thread 7fd189d9a640 thread_name:quiesce.agt 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fd19703e6f0] 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fd19708b94c] 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: 3: raise() 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: 4: abort() 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd197981fb6] 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd19798211a] 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ed9aa20fd2] 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ed9aa21091] 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ed9a9dba96] 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ed9a9db9f6] 2024-04-22T07:05:21.952 INFO:tasks.ceph.mds.c.smithi092.stderr: 11: ceph-mds(+0x48e3cf) [0x55ed9ab793cf] 2024-04-22T07:05:21.953 INFO:tasks.ceph.mds.c.smithi092.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ed9abc9c1b] 2024-04-22T07:05:21.953 INFO:tasks.ceph.mds.c.smithi092.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fd197089c02] 2024-04-22T07:05:21.953 INFO:tasks.ceph.mds.c.smithi092.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fd19710ec40] 2024-04-22T07:05:21.953 INFO:tasks.ceph.mds.c.smithi092.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:21.953 INFO:tasks.ceph.mds.c.smithi092.stderr: 2024-04-22T07:05:21.953 INFO:tasks.ceph.mds.a.smithi092.stderr: -4> 2024-04-22T07:05:21.906+0000 7f56a3c30640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f56a3c30640 time 2024-04-22T07:05:21.906600+0000 2024-04-22T07:05:21.953 INFO:tasks.ceph.mds.a.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:21.953 INFO:tasks.ceph.mds.a.smithi092.stderr: 2024-04-22T07:05:21.953 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.953 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f56b1781f5c] 2024-04-22T07:05:21.953 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f56b178211a] 2024-04-22T07:05:21.953 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56425b97afd2] 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x56425b97b091] 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56425b935a96] 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56425b9359f6] 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: ceph-mds(+0x48e3cf) [0x56425bad33cf] 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x56425bb23c1b] 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f56b0e89c02] 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f56b0f0ec40] 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: -3> 2024-04-22T07:05:21.907+0000 7f56a3c30640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: in thread 7f56a3c30640 thread_name:quiesce.agt 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:21.954 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f56b0e3e6f0] 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f56b0e8b94c] 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: raise() 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: abort() 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f56b1781fb6] 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f56b178211a] 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x56425b97afd2] 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x56425b97b091] 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56425b935a96] 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56425b9359f6] 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 11: ceph-mds(+0x48e3cf) [0x56425bad33cf] 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x56425bb23c1b] 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f56b0e89c02] 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f56b0f0ec40] 2024-04-22T07:05:21.955 INFO:tasks.ceph.mds.a.smithi092.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:21.956 INFO:tasks.ceph.mds.a.smithi092.stderr: 2024-04-22T07:05:22.387 INFO:tasks.ceph.mds.a.smithi092.stderr:daemon-helper: command crashed with signal 6 2024-04-22T07:05:22.396 INFO:tasks.ceph.mds.c.smithi092.stderr:daemon-helper: command crashed with signal 6 2024-04-22T07:05:22.411 INFO:tasks.ceph.mds.b.smithi145.stderr:daemon-helper: command crashed with signal 6 2024-04-22T07:05:22.411 INFO:tasks.ceph.mds.f.smithi145.stderr:daemon-helper: command crashed with signal 6 2024-04-22T07:05:26.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-22T07:05:26.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-22T07:05:26.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-22T07:05:26.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~0s 2024-04-22T07:05:33.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-04-22T07:05:33.461 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-04-22T07:05:33.461 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2024-04-22T07:05:33.461 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~7s 2024-04-22T07:05:35.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:05:35.225 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:05:35.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.577+0000 7f15617fa640 1 -- 172.21.15.92:0/3708777330 <== mds.0 v2:172.21.15.92:6838/3493022212 1 ==== command_reply(tid 0: -125 ) ==== 8+0+587 (crc 0 0 0) 0x559220d31f20 con 0x559220e199e0 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.577+0000 7f15617fa640 10 client.5091 handle_command_reply: tid=0 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.577+0000 7f158af77740 1 client.5091 shutdown 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.578+0000 7f158af77740 1 -- 172.21.15.92:0/3708777330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220de82d0 msgr2=0x559220dbdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.578+0000 7f158af77740 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220de82d0 0x559220dbdff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f157c08bbd0 tx=0x7f157c00a000 comp rx=0 tx=0).stop 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.578+0000 7f158af77740 1 -- 172.21.15.92:0/3708777330 shutdown_connections 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.578+0000 7f158af77740 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x559220e199e0 0x559220de6850 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.578+0000 7f158af77740 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x559220e02ba0 0x559220de6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.578+0000 7f158af77740 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559220dfe7d0 0x559220de5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.578+0000 7f15617fa640 0 client.5091 ms_handle_reset on v2:172.21.15.92:6838/3493022212 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.578+0000 7f158af77740 1 --2- 172.21.15.92:0/3708777330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559220de82d0 0x559220dbdff0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.578+0000 7f158af77740 1 -- 172.21.15.92:0/3708777330 >> 172.21.15.92:0/3708777330 conn(0x559220dbd810 msgr2=0x559220dd9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.578+0000 7f158af77740 1 -- 172.21.15.92:0/3708777330 shutdown_connections 2024-04-22T07:05:35.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.578+0000 7f158af77740 1 -- 172.21.15.92:0/3708777330 wait complete. 2024-04-22T07:05:35.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.578+0000 7f158af77740 20 client.5091 trim_cache size 0 max 16384 2024-04-22T07:05:35.578 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:05:35.578 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 18, 2024-04-22T07:05:35.578 INFO:teuthology.orchestra.run.smithi092.stdout: "leader": 4275, 2024-04-22T07:05:35.578 INFO:teuthology.orchestra.run.smithi092.stdout: "set_version": 4, 2024-04-22T07:05:35.578 INFO:teuthology.orchestra.run.smithi092.stdout: "sets": { 2024-04-22T07:05:35.578 INFO:teuthology.orchestra.run.smithi092.stdout: "2853aa63": { 2024-04-22T07:05:35.578 INFO:teuthology.orchestra.run.smithi092.stdout: "version": 4, 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: "age_ref": 0.0, 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: "state": { 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "CANCELED", 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: "timeout": 60.0, 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: "expiration": 79.7, 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: "members": { 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: "file:/": { 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: "excluded": true, 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: "state": { 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "QUIESCED", 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.9 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:35.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:35.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:35.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:35.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:35.580 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:05:35.580 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-04-22T07:05:35.581 INFO:teuthology.orchestra.run.smithi092.stderr:Error ECANCELED: 2024-04-22T07:05:35.581 WARNING:tasks.quiescer.fs.[cephfs]:Quiesce set got cancelled (cap = 10).Won't raise an error since this could be a failover, will wait for the next quiesce attempt 2024-04-22T07:05:35.581 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.7 2024-04-22T07:05:35.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:05:35.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.683+0000 7f1b5e9bd640 1 Processor -- start 2024-04-22T07:05:35.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.683+0000 7f1b5e9bd640 1 -- start start 2024-04-22T07:05:35.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.684+0000 7f1b5e9bd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b580d0b70 0x7f1b58154a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.684+0000 7f1b5e9bd640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b5815ab30 con 0x7f1b580d0f50 2024-04-22T07:05:35.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.684+0000 7f1b5e9bd640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b5815adf0 con 0x7f1b5815e9b0 2024-04-22T07:05:35.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.684+0000 7f1b5e9bd640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5815ba00 con 0x7f1b580d0b70 2024-04-22T07:05:35.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.684+0000 7f1b577fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1b580d0f50 0x7f1b5815e2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49888/0 (socket says 172.21.15.92:49888) 2024-04-22T07:05:35.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.684+0000 7f1b577fe640 1 -- 172.21.15.92:0/22206024 learned_addr learned my addr 172.21.15.92:0/22206024 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:05:35.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.684+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 369952124 0 0) 0x7f1b5815ab30 con 0x7f1b580d0f50 2024-04-22T07:05:35.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.684+0000 7f1b57fff640 1 --2- 172.21.15.92:0/22206024 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b580d0b70 0x7f1b58154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.684+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b40003520 con 0x7f1b580d0f50 2024-04-22T07:05:35.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.684+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1748526756 0 0) 0x7f1b40003520 con 0x7f1b580d0f50 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b580d0b70 msgr2=0x7f1b58154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b56ffd640 1 --2- 172.21.15.92:0/22206024 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b580d0b70 0x7f1b58154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 >> v1:172.21.15.145:6789/0 conn(0x7f1b5815e9b0 legacy=0x7f1b58165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5815b0d0 con 0x7f1b580d0f50 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7f1b4c001670 con 0x7f1b580d0f50 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 >> v1:172.21.15.92:6789/0 conn(0x7f1b580d0f50 legacy=0x7f1b5815e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b56ffd640 1 --2- 172.21.15.92:0/22206024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b400039e0 0x7f1b40003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b56ffd640 1 --2- 172.21.15.92:0/22206024 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b580d0b70 0x7f1b40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b56ffd640 1 --2- 172.21.15.92:0/22206024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b40004d40 0x7f1b400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5815ba00 con 0x7f1b40004d40 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4000baf0 con 0x7f1b400039e0 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1b4000bcf0 con 0x7f1b580d0b70 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b57fff640 1 --2- 172.21.15.92:0/22206024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b400039e0 0x7f1b40003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b5cf5a640 1 --2- 172.21.15.92:0/22206024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b40004d40 0x7f1b400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b577fe640 1 --2- 172.21.15.92:0/22206024 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b580d0b70 0x7f1b40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b5cf5a640 1 -- 172.21.15.92:0/22206024 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b580d0b70 msgr2=0x7f1b40004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:35.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b5cf5a640 1 --2- 172.21.15.92:0/22206024 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b580d0b70 0x7f1b40004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b5cf5a640 1 -- 172.21.15.92:0/22206024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b400039e0 msgr2=0x7f1b40003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:35.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b5cf5a640 1 --2- 172.21.15.92:0/22206024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b400039e0 0x7f1b40003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.685+0000 7f1b5cf5a640 1 -- 172.21.15.92:0/22206024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b40003520 con 0x7f1b40004d40 2024-04-22T07:05:35.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.686+0000 7f1b5cf5a640 1 --2- 172.21.15.92:0/22206024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b40004d40 0x7f1b400051c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1b50001e50 tx=0x7f1b40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8672de76a50d4735 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.686+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50001a70 con 0x7f1b40004d40 2024-04-22T07:05:35.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.686+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b50003b70 con 0x7f1b40004d40 2024-04-22T07:05:35.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.686+0000 7f1b56ffd640 1 -- 172.21.15.92:0/22206024 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b500050d0 con 0x7f1b40004d40 2024-04-22T07:05:35.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.687+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/22206024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b40004d40 msgr2=0x7f1b400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.687+0000 7f1b5e9bd640 1 --2- 172.21.15.92:0/22206024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b40004d40 0x7f1b400051c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1b50001e50 tx=0x7f1b40005830 comp rx=0 tx=0).stop 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.687+0000 7f1b5cf5a640 1 -- 172.21.15.92:0/22206024 reap_dead start 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.687+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/22206024 shutdown_connections 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.687+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/22206024 >> 172.21.15.92:0/22206024 conn(0x7f1b580736b0 msgr2=0x7f1b58073ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.687+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/22206024 shutdown_connections 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.687+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/22206024 wait complete. 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.688+0000 7f1b5e9bd640 1 Processor -- start 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.688+0000 7f1b5e9bd640 1 -- start start 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b5e9bd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b5814b060 0x7f1b58159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b5e9bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58157f50 0x7f1b58159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b5e9bd640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b5814b440 0x7f1b580cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b5e9bd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1b580cae00 con 0x7f1b58157f50 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b5e9bd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5815b690 con 0x7f1b5814b060 2024-04-22T07:05:35.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b5e9bd640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1b580caf80 con 0x7f1b5814b440 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b577fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58157f50 0x7f1b58159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b577fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58157f50 0x7f1b58159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43372/0 (socket says 172.21.15.92:43372) 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b577fe640 1 -- 172.21.15.92:0/2498780458 learned_addr learned my addr 172.21.15.92:0/2498780458 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b57fff640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b5814b060 0x7f1b58159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b5cf5a640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b5814b440 0x7f1b580cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b577fe640 1 -- 172.21.15.92:0/2498780458 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b5814b440 msgr2=0x7f1b580cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b577fe640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b5814b440 0x7f1b580cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b577fe640 1 -- 172.21.15.92:0/2498780458 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b5814b060 msgr2=0x7f1b58159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b577fe640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b5814b060 0x7f1b58159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b577fe640 1 -- 172.21.15.92:0/2498780458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b581f13e0 con 0x7f1b58157f50 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b577fe640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58157f50 0x7f1b58159ba0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c007290 tx=0x7f1b4c007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b54ff9640 1 -- 172.21.15.92:0/2498780458 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c00f9c0 con 0x7f1b58157f50 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b54ff9640 1 -- 172.21.15.92:0/2498780458 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4c018070 con 0x7f1b58157f50 2024-04-22T07:05:35.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/2498780458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b581f1680 con 0x7f1b58157f50 2024-04-22T07:05:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b54ff9640 1 -- 172.21.15.92:0/2498780458 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c00fdc0 con 0x7f1b58157f50 2024-04-22T07:05:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.689+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/2498780458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b58159140 con 0x7f1b58157f50 2024-04-22T07:05:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.690+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/2498780458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b580694d0 con 0x7f1b58157f50 2024-04-22T07:05:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.691+0000 7f1b54ff9640 1 -- 172.21.15.92:0/2498780458 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b4c01d460 con 0x7f1b58157f50 2024-04-22T07:05:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.691+0000 7f1b54ff9640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1b4406c8f0 0x7f1b4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.691+0000 7f1b54ff9640 1 -- 172.21.15.92:0/2498780458 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f1b4c08bdf0 con 0x7f1b58157f50 2024-04-22T07:05:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.691+0000 7f1b57fff640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1b4406c8f0 0x7f1b4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.692+0000 7f1b57fff640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1b4406c8f0 0x7f1b4406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009bc0 tx=0x7f1b48009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.700+0000 7f1b54ff9640 1 -- 172.21.15.92:0/2498780458 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b580694d0 con 0x7f1b58157f50 2024-04-22T07:05:35.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.942+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/2498780458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b58072c90 con 0x7f1b58157f50 2024-04-22T07:05:35.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.943+0000 7f1b54ff9640 1 -- 172.21.15.92:0/2498780458 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6178 (secure 0 0 0) 0x7f1b58072c90 con 0x7f1b58157f50 2024-04-22T07:05:35.942 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:05:35.942 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:03.129095+0000","modified":"2024-04-22T07:05:35.574694+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4275,"mds_1":4273,"mds_2":4269,"mds_3":4232,"mds_4":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"d","rank":4,"incarnation":18,"state":"up:replay","state_seq":1,"addr":"172.21.15.145:6835/1059009649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1059009649},{"type":"v1","addr":"172.21.15.145:6835","nonce":1059009649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4232":{"gid":4232,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.145:6837/3258302920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3258302920},{"type":"v1","addr":"172.21.15.145:6837","nonce":3258302920}]},"join_fscid":-1,"laggy_since":"2024-04-22T07:05:35.574403+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/1444537065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1444537065},{"type":"v1","addr":"172.21.15.92:6836","nonce":1444537065}]},"join_fscid":-1,"laggy_since":"2024-04-22T07:05:35.574407+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4273":{"gid":4273,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/101798872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":101798872},{"type":"v1","addr":"172.21.15.92:6837","nonce":101798872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6839/3493022212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3493022212},{"type":"v1","addr":"172.21.15.92:6839","nonce":3493022212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4232,4269,4273,4275]},"id":1}]} 2024-04-22T07:05:35.942 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 18 2024-04-22T07:05:35.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.945+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/2498780458 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1b4406c8f0 msgr2=0x7f1b4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.945+0000 7f1b5e9bd640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1b4406c8f0 0x7f1b4406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009bc0 tx=0x7f1b48009190 comp rx=0 tx=0).stop 2024-04-22T07:05:35.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.945+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/2498780458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58157f50 msgr2=0x7f1b58159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.945+0000 7f1b5e9bd640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58157f50 0x7f1b58159ba0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c007290 tx=0x7f1b4c007690 comp rx=0 tx=0).stop 2024-04-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.945+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/2498780458 shutdown_connections 2024-04-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.945+0000 7f1b5e9bd640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f1b4406c8f0 0x7f1b4406edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.945+0000 7f1b5e9bd640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1b5814b440 0x7f1b580cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.945+0000 7f1b5e9bd640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b58157f50 0x7f1b58159ba0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.945+0000 7f1b5e9bd640 1 --2- 172.21.15.92:0/2498780458 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b5814b060 0x7f1b58159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.945+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/2498780458 >> 172.21.15.92:0/2498780458 conn(0x7f1b580736b0 msgr2=0x7f1b580ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.945+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/2498780458 shutdown_connections 2024-04-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:35.945+0000 7f1b5e9bd640 1 -- 172.21.15.92:0/2498780458 wait complete. 2024-04-22T07:05:35.955 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.4275 quiesce db / --timeout 60 --expiration 79.7 --await' 2024-04-22T07:05:35.956 DEBUG:teuthology.orchestra.run.smithi092:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4275 quiesce db / --timeout 60 --expiration 79.7 --await 2024-04-22T07:05:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.063+0000 7fc995aeb640 1 Processor -- start 2024-04-22T07:05:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.063+0000 7fc995aeb640 1 -- start start 2024-04-22T07:05:36.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.064+0000 7fc995aeb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc9900cb220 0x7fc99015b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.064+0000 7fc995aeb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9900d0430 0x7fc9900d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.064+0000 7fc995aeb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9900cc690 con 0x7fc990155060 2024-04-22T07:05:36.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.064+0000 7fc995aeb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fc9900cb640 con 0x7fc9900d0430 2024-04-22T07:05:36.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.064+0000 7fc995aeb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc9900ccc50 con 0x7fc9900cb220 2024-04-22T07:05:36.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.064+0000 7fc98ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9900d0430 0x7fc9900d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.064+0000 7fc98ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9900d0430 0x7fc9900d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43384/0 (socket says 172.21.15.92:43384) 2024-04-22T07:05:36.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.064+0000 7fc98ffff640 1 -- 172.21.15.92:0/2362216350 learned_addr learned my addr 172.21.15.92:0/2362216350 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:36.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.064+0000 7fc98effd640 1 --2- 172.21.15.92:0/2362216350 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc9900cb220 0x7fc99015b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.064+0000 7fc98ffff640 1 -- 172.21.15.92:0/2362216350 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc9900cb220 msgr2=0x7fc99015b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:36.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.065+0000 7fc98ffff640 1 --2- 172.21.15.92:0/2362216350 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc9900cb220 0x7fc99015b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.065+0000 7fc98ffff640 1 -- 172.21.15.92:0/2362216350 >> v1:172.21.15.145:6789/0 conn(0x7fc990155060 legacy=0x7fc99015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:36.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.065+0000 7fc98ffff640 1 -- 172.21.15.92:0/2362216350 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9900cc970 con 0x7fc9900d0430 2024-04-22T07:05:36.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.065+0000 7fc98ffff640 1 --2- 172.21.15.92:0/2362216350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900d0430 0x7fc9900d08c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc988009e50 tx=0x7fc98802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70b9233752bfa0c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:36.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.065+0000 7fc98e7fc640 1 -- 172.21.15.92:0/2362216350 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98802cb90 con 0x7fc9900d0430 2024-04-22T07:05:36.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.066+0000 7fc98e7fc640 1 -- 172.21.15.92:0/2362216350 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98802cd30 con 0x7fc9900d0430 2024-04-22T07:05:36.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.066+0000 7fc98e7fc640 1 -- 172.21.15.92:0/2362216350 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988034610 con 0x7fc9900d0430 2024-04-22T07:05:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.066+0000 7fc995aeb640 1 -- 172.21.15.92:0/2362216350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900d0430 msgr2=0x7fc9900d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.066+0000 7fc995aeb640 1 --2- 172.21.15.92:0/2362216350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900d0430 0x7fc9900d08c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc988009e50 tx=0x7fc98802bff0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.066+0000 7fc995aeb640 1 -- 172.21.15.92:0/2362216350 shutdown_connections 2024-04-22T07:05:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.066+0000 7fc995aeb640 1 --2- 172.21.15.92:0/2362216350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900d0430 0x7fc9900d08c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.066+0000 7fc995aeb640 1 --2- 172.21.15.92:0/2362216350 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc9900cb220 0x7fc99015b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.066+0000 7fc995aeb640 1 -- 172.21.15.92:0/2362216350 >> 172.21.15.92:0/2362216350 conn(0x7fc9900723b0 msgr2=0x7fc99006cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.066+0000 7fc995aeb640 1 -- 172.21.15.92:0/2362216350 shutdown_connections 2024-04-22T07:05:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.066+0000 7fc995aeb640 1 -- 172.21.15.92:0/2362216350 wait complete. 2024-04-22T07:05:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.067+0000 7fc995aeb640 1 Processor -- start 2024-04-22T07:05:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.067+0000 7fc995aeb640 1 -- start start 2024-04-22T07:05:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.068+0000 7fc995aeb640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9900cb220 0x7fc990150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.068+0000 7fc995aeb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900d0430 0x7fc990150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.068+0000 7fc995aeb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc990155060 0x7fc9900c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.068+0000 7fc995aeb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc99015bb00 con 0x7fc9900d0430 2024-04-22T07:05:36.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.068+0000 7fc995aeb640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc99015b0a0 con 0x7fc9900cb220 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.068+0000 7fc995aeb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc9900c55b0 con 0x7fc990155060 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.068+0000 7fc98effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900d0430 0x7fc990150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.068+0000 7fc98effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900d0430 0x7fc990150cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43396/0 (socket says 172.21.15.92:43396) 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.068+0000 7fc98effd640 1 -- 172.21.15.92:0/1460443839 learned_addr learned my addr 172.21.15.92:0/1460443839 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.068+0000 7fc98f7fe640 1 --2- 172.21.15.92:0/1460443839 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9900cb220 0x7fc990150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.068+0000 7fc98ffff640 1 --2- 172.21.15.92:0/1460443839 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc990155060 0x7fc9900c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.068+0000 7fc98effd640 1 -- 172.21.15.92:0/1460443839 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc990155060 msgr2=0x7fc9900c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.069+0000 7fc98effd640 1 --2- 172.21.15.92:0/1460443839 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc990155060 0x7fc9900c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.069+0000 7fc98effd640 1 -- 172.21.15.92:0/1460443839 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9900cb220 msgr2=0x7fc990150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.069+0000 7fc98effd640 1 --2- 172.21.15.92:0/1460443839 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9900cb220 0x7fc990150250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.069+0000 7fc98effd640 1 -- 172.21.15.92:0/1460443839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9901f1bf0 con 0x7fc9900d0430 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.069+0000 7fc98effd640 1 --2- 172.21.15.92:0/1460443839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9900d0430 0x7fc990150cf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc98000e9e0 tx=0x7fc98000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:36.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.069+0000 7fc977fff640 1 -- 172.21.15.92:0/1460443839 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980017070 con 0x7fc9900d0430 2024-04-22T07:05:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.069+0000 7fc977fff640 1 -- 172.21.15.92:0/1460443839 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98000c9d0 con 0x7fc9900d0430 2024-04-22T07:05:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.069+0000 7fc995aeb640 1 -- 172.21.15.92:0/1460443839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc990150830 con 0x7fc9900d0430 2024-04-22T07:05:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.069+0000 7fc977fff640 1 -- 172.21.15.92:0/1460443839 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98000cdc0 con 0x7fc9900d0430 2024-04-22T07:05:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.069+0000 7fc995aeb640 1 -- 172.21.15.92:0/1460443839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901f1110 con 0x7fc9900d0430 2024-04-22T07:05:36.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.070+0000 7fc977fff640 1 -- 172.21.15.92:0/1460443839 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc98001a070 con 0x7fc9900d0430 2024-04-22T07:05:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.071+0000 7fc977fff640 1 --2- 172.21.15.92:0/1460443839 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc97c06c8f0 0x7fc97c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.071+0000 7fc98f7fe640 1 --2- 172.21.15.92:0/1460443839 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc97c06c8f0 0x7fc97c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.071+0000 7fc977fff640 1 -- 172.21.15.92:0/1460443839 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fc98008bd50 con 0x7fc9900d0430 2024-04-22T07:05:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.071+0000 7fc98f7fe640 1 --2- 172.21.15.92:0/1460443839 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc97c06c8f0 0x7fc97c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc984007fd0 tx=0x7fc984009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.072+0000 7fc9987bc740 1 Processor -- start 2024-04-22T07:05:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.072+0000 7fc9987bc740 1 -- start start 2024-04-22T07:05:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc9987bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b215e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc9987bc740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b6b080 0x5634a4b21b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc9987bc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b1fc60 0x5634a4b60b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc9987bc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5634a4a820b0 con 0x5634a4b1fc60 2024-04-22T07:05:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc9987bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5634a4a33680 con 0x5634a4b21200 2024-04-22T07:05:36.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc9987bc740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5634a4a3fc60 con 0x5634a4b6b080 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b1fc60 0x5634a4b60b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b1fc60 0x5634a4b60b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43406/0 (socket says 172.21.15.92:43406) 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc98ffff640 1 -- 172.21.15.92:0/865473986 learned_addr learned my addr 172.21.15.92:0/865473986 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc98f7fe640 1 --2- 172.21.15.92:0/865473986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b215e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc98effd640 1 --2- 172.21.15.92:0/865473986 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b6b080 0x5634a4b21b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc98ffff640 1 -- 172.21.15.92:0/865473986 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b6b080 msgr2=0x5634a4b21b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc98ffff640 1 --2- 172.21.15.92:0/865473986 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b6b080 0x5634a4b21b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc98ffff640 1 -- 172.21.15.92:0/865473986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 msgr2=0x5634a4b215e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc98ffff640 1 --2- 172.21.15.92:0/865473986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b215e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc98ffff640 1 -- 172.21.15.92:0/865473986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a4864c90 con 0x5634a4b1fc60 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc98f7fe640 1 --2- 172.21.15.92:0/865473986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b215e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc98ffff640 1 --2- 172.21.15.92:0/865473986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b1fc60 0x5634a4b60b80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc988036040 tx=0x7fc98802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227689e11ed2a4b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.073+0000 7fc9757fa640 1 -- 172.21.15.92:0/865473986 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98802cb90 con 0x5634a4b1fc60 2024-04-22T07:05:36.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.074+0000 7fc9757fa640 1 -- 172.21.15.92:0/865473986 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98802cd30 con 0x5634a4b1fc60 2024-04-22T07:05:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.074+0000 7fc9757fa640 1 -- 172.21.15.92:0/865473986 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98803e670 con 0x5634a4b1fc60 2024-04-22T07:05:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.074+0000 7fc9987bc740 1 -- 172.21.15.92:0/865473986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b1fc60 msgr2=0x5634a4b60b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.074+0000 7fc9987bc740 1 --2- 172.21.15.92:0/865473986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b1fc60 0x5634a4b60b80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc988036040 tx=0x7fc98802e000 comp rx=0 tx=0).stop 2024-04-22T07:05:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.074+0000 7fc9987bc740 1 -- 172.21.15.92:0/865473986 shutdown_connections 2024-04-22T07:05:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.074+0000 7fc9987bc740 1 --2- 172.21.15.92:0/865473986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b1fc60 0x5634a4b60b80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.074+0000 7fc9987bc740 1 --2- 172.21.15.92:0/865473986 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b6b080 0x5634a4b21b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.074+0000 7fc9987bc740 1 --2- 172.21.15.92:0/865473986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b215e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.074+0000 7fc9987bc740 1 -- 172.21.15.92:0/865473986 >> 172.21.15.92:0/865473986 conn(0x5634a4b1dc00 msgr2=0x5634a4b202b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.074+0000 7fc9987bc740 1 -- 172.21.15.92:0/865473986 shutdown_connections 2024-04-22T07:05:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.074+0000 7fc9987bc740 1 -- 172.21.15.92:0/865473986 wait complete. 2024-04-22T07:05:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.075+0000 7fc9987bc740 1 -- start start 2024-04-22T07:05:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.075+0000 7fc9987bc740 1 Processor -- start 2024-04-22T07:05:36.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc9987bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b71930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc9987bc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 0x5634a4b426d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc9987bc740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b61e80 0x5634a4b622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc9987bc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5634a4a85400 con 0x5634a4b6b080 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc9987bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5634a4a41840 con 0x5634a4b21200 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc9987bc740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5634a4a8a720 con 0x5634a4b61e80 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc9987bc740 10 client.0 _ll_register_callbacks cb 0x5634a4a92ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc98effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 0x5634a4b426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc98effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 0x5634a4b426d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43412/0 (socket says 172.21.15.92:43412) 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc98effd640 1 -- 172.21.15.92:0/1208230006 learned_addr learned my addr 172.21.15.92:0/1208230006 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc98f7fe640 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b71930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc98ffff640 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b61e80 0x5634a4b622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc98effd640 1 -- 172.21.15.92:0/1208230006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b61e80 msgr2=0x5634a4b622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc98effd640 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b61e80 0x5634a4b622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc98effd640 1 -- 172.21.15.92:0/1208230006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 msgr2=0x5634a4b71930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc98effd640 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b71930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc98effd640 1 -- 172.21.15.92:0/1208230006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a48c5420 con 0x5634a4b6b080 2024-04-22T07:05:36.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc98ffff640 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b61e80 0x5634a4b622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc98effd640 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 0x5634a4b426d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc98008a0b0 tx=0x7fc98005a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f78e91ff8598f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc96e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc96e7fc640 1 -- 172.21.15.92:0/1208230006 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980021050 con 0x5634a4b6b080 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc96e7fc640 1 -- 172.21.15.92:0/1208230006 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fc98005adb0 con 0x5634a4b6b080 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc9987bc740 1 -- 172.21.15.92:0/1208230006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98802cd30 con 0x5634a4b6b080 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc96e7fc640 1 -- 172.21.15.92:0/1208230006 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc980058cd0 con 0x5634a4b6b080 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc9987bc740 10 client.5109 fetch_fsmap learned FSMap version 18 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.076+0000 7fc9987bc740 1 -- 172.21.15.92:0/1208230006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5634a4758580 con 0x5634a4b6b080 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.077+0000 7fc96e7fc640 1 -- 172.21.15.92:0/1208230006 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98802cd30 con 0x5634a4b6b080 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.077+0000 7fc96e7fc640 1 -- 172.21.15.92:0/1208230006 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 18) ==== 4171+0+0 (secure 0 0 0) 0x7fc980024060 con 0x5634a4b6b080 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.077+0000 7fc9987bc740 10 client.5109 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.077+0000 7fc9987bc740 10 client.5109 resolve_mds: validated gid 4275 aka daemon mds.e 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.077+0000 7fc9987bc740 20 client.5109 populate_metadata read hostname 'smithi092' 2024-04-22T07:05:36.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.077+0000 7fc9987bc740 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x5634a4b49380 0x5634a4b5c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.077+0000 7fc9987bc740 4 client.5109 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.077+0000 7fc9987bc740 1 -- 172.21.15.92:0/1208230006 --> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634a4a93b10 con 0x5634a4b49380 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.077+0000 7fc98f7fe640 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x5634a4b49380 0x5634a4b5c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.077+0000 7fc96e7fc640 10 client.5109 ms_handle_connect on v2:172.21.15.92:6838/3493022212 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.077+0000 7fc98f7fe640 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x5634a4b49380 0x5634a4b5c0a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e20de2da7323444 server_cookie=1c018360deaafb6b in_seq=0 out_seq=0 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc96e7fc640 1 -- 172.21.15.92:0/1208230006 <== mds.0 v2:172.21.15.92:6838/3493022212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634a4a93b10 con 0x5634a4b49380 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc96e7fc640 10 client.5109 handle_command_reply: tid=0 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc9987bc740 1 client.5109 shutdown 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc9987bc740 1 -- 172.21.15.92:0/1208230006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 msgr2=0x5634a4b426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc9987bc740 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 0x5634a4b426d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc98008a0b0 tx=0x7fc98005a0c0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc9987bc740 1 -- 172.21.15.92:0/1208230006 shutdown_connections 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc9987bc740 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x5634a4b49380 0x5634a4b5c0a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc9987bc740 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b61e80 0x5634a4b622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc9987bc740 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 0x5634a4b426d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc9987bc740 1 --2- 172.21.15.92:0/1208230006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b71930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc9987bc740 1 -- 172.21.15.92:0/1208230006 >> 172.21.15.92:0/1208230006 conn(0x5634a4b1fc60 msgr2=0x5634a4b1f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:36.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc96e7fc640 0 client.5109 ms_handle_reset on v2:172.21.15.92:6838/3493022212 2024-04-22T07:05:36.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc9987bc740 1 -- 172.21.15.92:0/1208230006 shutdown_connections 2024-04-22T07:05:36.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc9987bc740 1 -- 172.21.15.92:0/1208230006 wait complete. 2024-04-22T07:05:36.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.078+0000 7fc9987bc740 20 client.5109 trim_cache size 0 max 16384 2024-04-22T07:05:36.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.095+0000 7fc9987bc740 1 Processor -- start 2024-04-22T07:05:36.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.095+0000 7fc9987bc740 1 -- start start 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.096+0000 7fc9987bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b5c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.096+0000 7fc9987bc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b61e80 0x5634a4b202b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.096+0000 7fc9987bc740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b6b080 0x5634a4b207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.096+0000 7fc9987bc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901fdb80 con 0x5634a4b61e80 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.096+0000 7fc9987bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901f4b20 con 0x5634a4b21200 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.096+0000 7fc9987bc740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc9901fd4b0 con 0x5634a4b6b080 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.096+0000 7fc98effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b61e80 0x5634a4b202b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.096+0000 7fc98effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b61e80 0x5634a4b202b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43418/0 (socket says 172.21.15.92:43418) 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.096+0000 7fc98effd640 1 -- 172.21.15.92:0/458985141 learned_addr learned my addr 172.21.15.92:0/458985141 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.096+0000 7fc98f7fe640 1 --2- 172.21.15.92:0/458985141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b5c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.096+0000 7fc98ffff640 1 --2- 172.21.15.92:0/458985141 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b6b080 0x5634a4b207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.097+0000 7fc98effd640 1 -- 172.21.15.92:0/458985141 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b6b080 msgr2=0x5634a4b207f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.097+0000 7fc98effd640 1 --2- 172.21.15.92:0/458985141 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b6b080 0x5634a4b207f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.097+0000 7fc98effd640 1 -- 172.21.15.92:0/458985141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 msgr2=0x5634a4b5c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.097+0000 7fc98effd640 1 --2- 172.21.15.92:0/458985141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b5c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.097+0000 7fc98effd640 1 -- 172.21.15.92:0/458985141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a4b22060 con 0x5634a4b61e80 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.097+0000 7fc98effd640 1 --2- 172.21.15.92:0/458985141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b61e80 0x5634a4b202b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc980056200 tx=0x7fc98005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c03f3b75f112fb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.097+0000 7fc974ff9640 1 -- 172.21.15.92:0/458985141 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980021030 con 0x5634a4b61e80 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.097+0000 7fc974ff9640 1 -- 172.21.15.92:0/458985141 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc980057920 con 0x5634a4b61e80 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.097+0000 7fc974ff9640 1 -- 172.21.15.92:0/458985141 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98005adb0 con 0x5634a4b61e80 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.098+0000 7fc9987bc740 1 -- 172.21.15.92:0/458985141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b61e80 msgr2=0x5634a4b202b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.098+0000 7fc9987bc740 1 --2- 172.21.15.92:0/458985141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b61e80 0x5634a4b202b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc980056200 tx=0x7fc98005b000 comp rx=0 tx=0).stop 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.098+0000 7fc9987bc740 1 -- 172.21.15.92:0/458985141 shutdown_connections 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.098+0000 7fc9987bc740 1 --2- 172.21.15.92:0/458985141 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b6b080 0x5634a4b207f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.098+0000 7fc9987bc740 1 --2- 172.21.15.92:0/458985141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b61e80 0x5634a4b202b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.098+0000 7fc9987bc740 1 --2- 172.21.15.92:0/458985141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b21200 0x5634a4b5c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.098+0000 7fc9987bc740 1 -- 172.21.15.92:0/458985141 >> 172.21.15.92:0/458985141 conn(0x5634a4b1fc60 msgr2=0x5634a4b622c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:36.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.098+0000 7fc9987bc740 1 -- 172.21.15.92:0/458985141 shutdown_connections 2024-04-22T07:05:36.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.098+0000 7fc9987bc740 1 -- 172.21.15.92:0/458985141 wait complete. 2024-04-22T07:05:36.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.098+0000 7fc9987bc740 1 -- start start 2024-04-22T07:05:36.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.098+0000 7fc9987bc740 1 Processor -- start 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.099+0000 7fc9987bc740 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b21200 0x5634a4b206b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.099+0000 7fc9987bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b61e80 0x5634a4bdf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.099+0000 7fc9987bc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 0x5634a4bdf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.099+0000 7fc9987bc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901fcde0 con 0x5634a4b6b080 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.099+0000 7fc9987bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901fc740 con 0x5634a4b61e80 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.099+0000 7fc9987bc740 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5634a4a93300 con 0x5634a4b21200 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.099+0000 7fc9987bc740 10 client.0 _ll_register_callbacks cb 0x7fc9800096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.099+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 0x5634a4bdf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.099+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 0x5634a4bdf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43426/0 (socket says 172.21.15.92:43426) 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.099+0000 7fc98ffff640 1 -- 172.21.15.92:0/3848257493 learned_addr learned my addr 172.21.15.92:0/3848257493 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.099+0000 7fc98effd640 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b61e80 0x5634a4bdf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.099+0000 7fc98f7fe640 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b21200 0x5634a4b206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc98ffff640 1 -- 172.21.15.92:0/3848257493 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b21200 msgr2=0x5634a4b206b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc98ffff640 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b21200 0x5634a4b206b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc98ffff640 1 -- 172.21.15.92:0/3848257493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b61e80 msgr2=0x5634a4bdf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc98ffff640 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b61e80 0x5634a4bdf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc98ffff640 1 -- 172.21.15.92:0/3848257493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a4b1ca40 con 0x5634a4b6b080 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc98ffff640 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 0x5634a4bdf6d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc988009f40 tx=0x7fc988004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facd5e2b9691901e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc96e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc96e7fc640 1 -- 172.21.15.92:0/3848257493 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98802c670 con 0x5634a4b6b080 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc96e7fc640 1 -- 172.21.15.92:0/3848257493 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fc988004100 con 0x5634a4b6b080 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc9987bc740 1 -- 172.21.15.92:0/3848257493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc980057920 con 0x5634a4b6b080 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc96e7fc640 1 -- 172.21.15.92:0/3848257493 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc988058050 con 0x5634a4b6b080 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc9987bc740 10 client.5115 fetch_fsmap learned FSMap version 18 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc9987bc740 1 -- 172.21.15.92:0/3848257493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5634a4b5f5e0 con 0x5634a4b6b080 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc96e7fc640 1 -- 172.21.15.92:0/3848257493 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc980057920 con 0x5634a4b6b080 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc96e7fc640 1 -- 172.21.15.92:0/3848257493 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 18) ==== 4171+0+0 (secure 0 0 0) 0x7fc98805b020 con 0x5634a4b6b080 2024-04-22T07:05:36.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc9987bc740 10 client.5115 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc9987bc740 10 client.5115 resolve_mds: validated gid 4275 aka daemon mds.e 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc9987bc740 20 client.5115 populate_metadata read hostname 'smithi092' 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc9987bc740 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x5634a4b483f0 0x5634a4bdfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc9987bc740 4 client.5115 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.7, "await": true}] 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.100+0000 7fc9987bc740 1 -- 172.21.15.92:0/3848257493 --> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.7, "await": true}) -- 0x5634a4a8d980 con 0x5634a4b483f0 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.101+0000 7fc98f7fe640 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x5634a4b483f0 0x5634a4bdfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.101+0000 7fc96e7fc640 10 client.5115 ms_handle_connect on v2:172.21.15.92:6838/3493022212 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.101+0000 7fc98f7fe640 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x5634a4b483f0 0x5634a4bdfc10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c11d64bbdd3419a server_cookie=9169b2259eb9100a in_seq=0 out_seq=0 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.101+0000 7fc96e7fc640 1 -- 172.21.15.92:0/3848257493 <== mds.0 v2:172.21.15.92:6838/3493022212 1 ==== command_reply(tid 0: -9 ) ==== 8+0+583 (crc 0 0 0) 0x5634a4a8d980 con 0x5634a4b483f0 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.101+0000 7fc96e7fc640 10 client.5115 handle_command_reply: tid=0 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.101+0000 7fc9987bc740 1 client.5115 shutdown 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.102+0000 7fc9987bc740 1 -- 172.21.15.92:0/3848257493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 msgr2=0x5634a4bdf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.102+0000 7fc9987bc740 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 0x5634a4bdf6d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc988009f40 tx=0x7fc988004ab0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.102+0000 7fc9987bc740 1 -- 172.21.15.92:0/3848257493 shutdown_connections 2024-04-22T07:05:36.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.102+0000 7fc9987bc740 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.92:6838/3493022212,v1:172.21.15.92:6839/3493022212] conn(0x5634a4b483f0 0x5634a4bdfc10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.102+0000 7fc9987bc740 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634a4b6b080 0x5634a4bdf6d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.102+0000 7fc9987bc740 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5634a4b61e80 0x5634a4bdf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.102+0000 7fc9987bc740 1 --2- 172.21.15.92:0/3848257493 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5634a4b21200 0x5634a4b206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.102+0000 7fc96e7fc640 0 client.5115 ms_handle_reset on v2:172.21.15.92:6838/3493022212 2024-04-22T07:05:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.102+0000 7fc9987bc740 1 -- 172.21.15.92:0/3848257493 >> 172.21.15.92:0/3848257493 conn(0x5634a4b1fc60 msgr2=0x5634a4b1ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.102+0000 7fc9987bc740 1 -- 172.21.15.92:0/3848257493 shutdown_connections 2024-04-22T07:05:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.102+0000 7fc9987bc740 1 -- 172.21.15.92:0/3848257493 wait complete. 2024-04-22T07:05:36.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:36.102+0000 7fc9987bc740 20 client.5115 trim_cache size 0 max 16384 2024-04-22T07:05:36.106 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-22T07:05:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 18, 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "leader": 4275, 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "set_version": 6, 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "sets": { 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "66d766a4": { 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "version": 6, 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "age_ref": 0.0, 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "state": { 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FAILED", 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "timeout": 60.0, 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "expiration": 79.7, 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "members": { 2024-04-22T07:05:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "file:/": { 2024-04-22T07:05:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: "excluded": false, 2024-04-22T07:05:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: "state": { 2024-04-22T07:05:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FAILED", 2024-04-22T07:05:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0 2024-04-22T07:05:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-22T07:05:36.108 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-22T07:05:36.109 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-22T07:05:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:Error EBADF: 2024-04-22T07:05:36.109 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't quiesce root with rc: -9 (EBADF), stdout: { "epoch": 18, "leader": 4275, "set_version": 6, "sets": { "66d766a4": { "version": 6, "age_ref": 0.0, "state": { "name": "FAILED", "age": 0.0 }, "timeout": 60.0, "expiration": 79.7, "members": { "file:/": { "excluded": false, "state": { "name": "FAILED", "age": 0.0 } } } } } } 2024-04-22T07:05:36.109 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 240, in _run self.do_quiesce(d) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 197, in do_quiesce raise RuntimeError(f"Error quiescing root: {rcinfo}") RuntimeError: Error quiescing root: -9 (EBADF) 2024-04-22T07:05:39.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2024-04-22T07:05:39.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2024-04-22T07:05:39.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2024-04-22T07:05:39.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~13s 2024-04-22T07:05:39.975 INFO:tasks.daemonwatchdog.daemon_watchdog:quiescer.fs.[cephfs] failed 2024-04-22T07:05:39.976 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-04-22T07:05:39.976 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:05:39.976 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T07:05:39.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:05:40.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.141+0000 7f19dbd59640 1 Processor -- start 2024-04-22T07:05:40.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.141+0000 7f19dbd59640 1 -- start start 2024-04-22T07:05:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.142+0000 7f19dbd59640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19d40d0f50 0x7f19d415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.142+0000 7f19dbd59640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19d415ab30 con 0x7f19d40d0b70 2024-04-22T07:05:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.142+0000 7f19dbd59640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19d415adf0 con 0x7f19d40cb120 2024-04-22T07:05:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.142+0000 7f19dbd59640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f19d415ba00 con 0x7f19d40d0f50 2024-04-22T07:05:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.142+0000 7f19d92f4640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19d40d0f50 0x7f19d415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.142+0000 7f19d9af5640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f19d40d0b70 0x7f19d4154a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50812/0 (socket says 172.21.15.92:50812) 2024-04-22T07:05:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.142+0000 7f19d9af5640 1 -- 172.21.15.92:0/3509087510 learned_addr learned my addr 172.21.15.92:0/3509087510 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:05:40.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.143+0000 7f19d92f4640 1 -- 172.21.15.92:0/3509087510 >> v1:172.21.15.145:6789/0 conn(0x7f19d40cb120 legacy=0x7f19d4165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.143+0000 7f19d92f4640 1 -- 172.21.15.92:0/3509087510 >> v1:172.21.15.92:6789/0 conn(0x7f19d40d0b70 legacy=0x7f19d4154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.143+0000 7f19d92f4640 1 -- 172.21.15.92:0/3509087510 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d415b0d0 con 0x7f19d40d0f50 2024-04-22T07:05:40.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.144+0000 7f19d92f4640 1 --2- 172.21.15.92:0/3509087510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40d0f50 0x7f19d415e2a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f19c40097a0 tx=0x7f19c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941530fba6333ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:40.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.144+0000 7f19d8af3640 1 -- 172.21.15.92:0/3509087510 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4035020 con 0x7f19d40d0f50 2024-04-22T07:05:40.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.144+0000 7f19d8af3640 1 -- 172.21.15.92:0/3509087510 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c4031830 con 0x7f19d40d0f50 2024-04-22T07:05:40.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.144+0000 7f19d8af3640 1 -- 172.21.15.92:0/3509087510 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4031c00 con 0x7f19d40d0f50 2024-04-22T07:05:40.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.144+0000 7f19dbd59640 1 -- 172.21.15.92:0/3509087510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40d0f50 msgr2=0x7f19d415e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.144+0000 7f19dbd59640 1 --2- 172.21.15.92:0/3509087510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40d0f50 0x7f19d415e2a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f19c40097a0 tx=0x7f19c402b8d0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.144+0000 7f19dbd59640 1 -- 172.21.15.92:0/3509087510 shutdown_connections 2024-04-22T07:05:40.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.145+0000 7f19dbd59640 1 --2- 172.21.15.92:0/3509087510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40d0f50 0x7f19d415e2a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.145+0000 7f19dbd59640 1 -- 172.21.15.92:0/3509087510 >> 172.21.15.92:0/3509087510 conn(0x7f19d40736b0 msgr2=0x7f19d4073ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:40.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.145+0000 7f19dbd59640 1 -- 172.21.15.92:0/3509087510 shutdown_connections 2024-04-22T07:05:40.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.145+0000 7f19dbd59640 1 -- 172.21.15.92:0/3509087510 wait complete. 2024-04-22T07:05:40.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.146+0000 7f19dbd59640 1 Processor -- start 2024-04-22T07:05:40.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.146+0000 7f19dbd59640 1 -- start start 2024-04-22T07:05:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.146+0000 7f19dbd59640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19d40cb120 0x7f19d4153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.146+0000 7f19dbd59640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19d40d0b70 0x7f19d40c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.146+0000 7f19dbd59640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40d0f50 0x7f19d40ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.146+0000 7f19dbd59640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19d40cae00 con 0x7f19d40d0f50 2024-04-22T07:05:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.146+0000 7f19dbd59640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f19d415b690 con 0x7f19d40d0b70 2024-04-22T07:05:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19dbd59640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f19d40caf80 con 0x7f19d40cb120 2024-04-22T07:05:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19da2f6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40d0f50 0x7f19d40ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19da2f6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40d0f50 0x7f19d40ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32956/0 (socket says 172.21.15.92:32956) 2024-04-22T07:05:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19da2f6640 1 -- 172.21.15.92:0/103215929 learned_addr learned my addr 172.21.15.92:0/103215929 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19d9af5640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19d40cb120 0x7f19d4153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19d92f4640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19d40d0b70 0x7f19d40c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19da2f6640 1 -- 172.21.15.92:0/103215929 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19d40cb120 msgr2=0x7f19d4153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19da2f6640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19d40cb120 0x7f19d4153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19da2f6640 1 -- 172.21.15.92:0/103215929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19d40d0b70 msgr2=0x7f19d40c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19da2f6640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19d40d0b70 0x7f19d40c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19da2f6640 1 -- 172.21.15.92:0/103215929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d4152b90 con 0x7f19d40d0f50 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19da2f6640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40d0f50 0x7f19d40ca260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00c740 tx=0x7f19cc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19ca7fc640 1 -- 172.21.15.92:0/103215929 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc016020 con 0x7f19d40d0f50 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19ca7fc640 1 -- 172.21.15.92:0/103215929 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19cc00ce50 con 0x7f19d40d0f50 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19ca7fc640 1 -- 172.21.15.92:0/103215929 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc003970 con 0x7f19d40d0f50 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.147+0000 7f19dbd59640 1 -- 172.21.15.92:0/103215929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d41f4220 con 0x7f19d40d0f50 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.148+0000 7f19dbd59640 1 -- 172.21.15.92:0/103215929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d41f47e0 con 0x7f19d40d0f50 2024-04-22T07:05:40.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.149+0000 7f19dbd59640 1 -- 172.21.15.92:0/103215929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d40b6670 con 0x7f19d40d0f50 2024-04-22T07:05:40.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.150+0000 7f19ca7fc640 1 -- 172.21.15.92:0/103215929 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19cc003b10 con 0x7f19d40d0f50 2024-04-22T07:05:40.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.150+0000 7f19ca7fc640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f19b806c920 0x7f19b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.150+0000 7f19ca7fc640 1 -- 172.21.15.92:0/103215929 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f19cc08b950 con 0x7f19d40d0f50 2024-04-22T07:05:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.150+0000 7f19d9af5640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f19b806c920 0x7f19b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.151+0000 7f19d9af5640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f19b806c920 0x7f19b806ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f19d0005880 tx=0x7f19d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.155+0000 7f19ca7fc640 1 -- 172.21.15.92:0/103215929 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19d40b6670 con 0x7f19d40d0f50 2024-04-22T07:05:40.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.374+0000 7f19dbd59640 1 -- 172.21.15.92:0/103215929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19d4069930 con 0x7f19d40d0f50 2024-04-22T07:05:40.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.375+0000 7f19ca7fc640 1 -- 172.21.15.92:0/103215929 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16881 (secure 0 0 0) 0x7f19cc020020 con 0x7f19d40d0f50 2024-04-22T07:05:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:05:40.373 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:05:35.574514+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:6836/1058270486":"2024-04-23T07:05:35.574446+0000","172.21.15.145:6833/1058270486":"2024-04-23T07:05:35.574446+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:05:40.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.378+0000 7f19dbd59640 1 -- 172.21.15.92:0/103215929 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f19b806c920 msgr2=0x7f19b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.378+0000 7f19dbd59640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f19b806c920 0x7f19b806ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f19d0005880 tx=0x7f19d0009040 comp rx=0 tx=0).stop 2024-04-22T07:05:40.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.378+0000 7f19dbd59640 1 -- 172.21.15.92:0/103215929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40d0f50 msgr2=0x7f19d40ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.378+0000 7f19dbd59640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40d0f50 0x7f19d40ca260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00c740 tx=0x7f19cc00cc00 comp rx=0 tx=0).stop 2024-04-22T07:05:40.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.378+0000 7f19dbd59640 1 -- 172.21.15.92:0/103215929 shutdown_connections 2024-04-22T07:05:40.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.378+0000 7f19dbd59640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f19b806c920 0x7f19b806ede0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.378+0000 7f19dbd59640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40d0f50 0x7f19d40ca260 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.378+0000 7f19dbd59640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19d40d0b70 0x7f19d40c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.378+0000 7f19dbd59640 1 --2- 172.21.15.92:0/103215929 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19d40cb120 0x7f19d4153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.378+0000 7f19dbd59640 1 -- 172.21.15.92:0/103215929 >> 172.21.15.92:0/103215929 conn(0x7f19d40736b0 msgr2=0x7f19d40ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:40.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.378+0000 7f19dbd59640 1 -- 172.21.15.92:0/103215929 shutdown_connections 2024-04-22T07:05:40.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.378+0000 7f19dbd59640 1 -- 172.21.15.92:0/103215929 wait complete. 2024-04-22T07:05:40.387 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:03:57.947601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:01.058111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:02.085602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:05:40.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:05:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.498+0000 7ffaedaeb640 1 Processor -- start 2024-04-22T07:05:40.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.498+0000 7ffaedaeb640 1 -- start start 2024-04-22T07:05:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.498+0000 7ffaedaeb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffae80ce380 0x7ffae80ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.498+0000 7ffaedaeb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffae8161350 0x7ffae8163740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.498+0000 7ffaedaeb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffae80c54a0 con 0x7ffae8070db0 2024-04-22T07:05:40.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.499+0000 7ffaedaeb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7ffae80ca4d0 con 0x7ffae80ce380 2024-04-22T07:05:40.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.499+0000 7ffaedaeb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffae80c8e90 con 0x7ffae8161350 2024-04-22T07:05:40.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.499+0000 7ffae77fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffae80ce380 0x7ffae80ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.499+0000 7ffae6ffd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ffae8070db0 0x7ffae80c3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50828/0 (socket says 172.21.15.92:50828) 2024-04-22T07:05:40.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.499+0000 7ffae6ffd640 1 -- 172.21.15.92:0/2455841485 learned_addr learned my addr 172.21.15.92:0/2455841485 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:05:40.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.499+0000 7ffae77fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffae80ce380 0x7ffae80ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32970/0 (socket says 172.21.15.92:32970) 2024-04-22T07:05:40.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.499+0000 7ffae7fff640 1 --2- 172.21.15.92:0/2455841485 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffae8161350 0x7ffae8163740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.499+0000 7ffae77fe640 1 -- 172.21.15.92:0/2455841485 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffae8161350 msgr2=0x7ffae8163740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.499+0000 7ffae77fe640 1 --2- 172.21.15.92:0/2455841485 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffae8161350 0x7ffae8163740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.499+0000 7ffae77fe640 1 -- 172.21.15.92:0/2455841485 >> v1:172.21.15.92:6789/0 conn(0x7ffae8070db0 legacy=0x7ffae80c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.499+0000 7ffae77fe640 1 -- 172.21.15.92:0/2455841485 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae80c9170 con 0x7ffae80ce380 2024-04-22T07:05:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.500+0000 7ffae77fe640 1 --2- 172.21.15.92:0/2455841485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae80ce380 0x7ffae80ce760 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffad80097a0 tx=0x7ffad802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57704269b8a5214 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.500+0000 7ffae67fc640 1 -- 172.21.15.92:0/2455841485 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8031780 con 0x7ffae80ce380 2024-04-22T07:05:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.500+0000 7ffae67fc640 1 -- 172.21.15.92:0/2455841485 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad8037070 con 0x7ffae80ce380 2024-04-22T07:05:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.500+0000 7ffae67fc640 1 -- 172.21.15.92:0/2455841485 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8031b40 con 0x7ffae80ce380 2024-04-22T07:05:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.501+0000 7ffaedaeb640 1 -- 172.21.15.92:0/2455841485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae80ce380 msgr2=0x7ffae80ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.501+0000 7ffaedaeb640 1 --2- 172.21.15.92:0/2455841485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae80ce380 0x7ffae80ce760 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffad80097a0 tx=0x7ffad802b820 comp rx=0 tx=0).stop 2024-04-22T07:05:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.501+0000 7ffaedaeb640 1 -- 172.21.15.92:0/2455841485 shutdown_connections 2024-04-22T07:05:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.501+0000 7ffaedaeb640 1 --2- 172.21.15.92:0/2455841485 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffae8161350 0x7ffae8163740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.501+0000 7ffaedaeb640 1 --2- 172.21.15.92:0/2455841485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae80ce380 0x7ffae80ce760 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.501+0000 7ffaedaeb640 1 -- 172.21.15.92:0/2455841485 >> 172.21.15.92:0/2455841485 conn(0x7ffae806db80 msgr2=0x7ffae806df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.501+0000 7ffaedaeb640 1 -- 172.21.15.92:0/2455841485 shutdown_connections 2024-04-22T07:05:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.501+0000 7ffaedaeb640 1 -- 172.21.15.92:0/2455841485 wait complete. 2024-04-22T07:05:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.502+0000 7ffaedaeb640 1 Processor -- start 2024-04-22T07:05:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.502+0000 7ffaedaeb640 1 -- start start 2024-04-22T07:05:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffaedaeb640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffae8070db0 0x7ffae80c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffaedaeb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae80ce380 0x7ffae80cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffaedaeb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffae8161350 0x7ffae815a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffaedaeb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8070260 con 0x7ffae80ce380 2024-04-22T07:05:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffaedaeb640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8159740 con 0x7ffae8070db0 2024-04-22T07:05:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffaedaeb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffae80c5190 con 0x7ffae8161350 2024-04-22T07:05:40.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffae6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae80ce380 0x7ffae80cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffae6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae80ce380 0x7ffae80cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32978/0 (socket says 172.21.15.92:32978) 2024-04-22T07:05:40.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffae6ffd640 1 -- 172.21.15.92:0/4198071450 learned_addr learned my addr 172.21.15.92:0/4198071450 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:40.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffae77fe640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffae8070db0 0x7ffae80c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffae7fff640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffae8161350 0x7ffae815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffae6ffd640 1 -- 172.21.15.92:0/4198071450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffae8161350 msgr2=0x7ffae815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffae6ffd640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffae8161350 0x7ffae815a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffae6ffd640 1 -- 172.21.15.92:0/4198071450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffae8070db0 msgr2=0x7ffae80c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffae6ffd640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffae8070db0 0x7ffae80c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.503+0000 7ffae6ffd640 1 -- 172.21.15.92:0/4198071450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae8156a50 con 0x7ffae80ce380 2024-04-22T07:05:40.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.504+0000 7ffae77fe640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffae8070db0 0x7ffae80c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:40.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.504+0000 7ffae6ffd640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae80ce380 0x7ffae80cb440 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00c3f0 tx=0x7ffadc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:40.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.504+0000 7ffacffff640 1 -- 172.21.15.92:0/4198071450 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc017070 con 0x7ffae80ce380 2024-04-22T07:05:40.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.504+0000 7ffacffff640 1 -- 172.21.15.92:0/4198071450 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffadc005ce0 con 0x7ffae80ce380 2024-04-22T07:05:40.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.504+0000 7ffaedaeb640 1 -- 172.21.15.92:0/4198071450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae81ee840 con 0x7ffae80ce380 2024-04-22T07:05:40.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.504+0000 7ffacffff640 1 -- 172.21.15.92:0/4198071450 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc007310 con 0x7ffae80ce380 2024-04-22T07:05:40.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.504+0000 7ffaedaeb640 1 -- 172.21.15.92:0/4198071450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae81eee00 con 0x7ffae80ce380 2024-04-22T07:05:40.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.505+0000 7ffaedaeb640 1 -- 172.21.15.92:0/4198071450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae8068b00 con 0x7ffae80ce380 2024-04-22T07:05:40.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.506+0000 7ffacffff640 1 -- 172.21.15.92:0/4198071450 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffadc01a070 con 0x7ffae80ce380 2024-04-22T07:05:40.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.506+0000 7ffacffff640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ffad406c8f0 0x7ffad406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.506+0000 7ffacffff640 1 -- 172.21.15.92:0/4198071450 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7ffadc08af30 con 0x7ffae80ce380 2024-04-22T07:05:40.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.506+0000 7ffae77fe640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ffad406c8f0 0x7ffad406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.507+0000 7ffae77fe640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ffad406c8f0 0x7ffad406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffad8036000 tx=0x7ffad803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:40.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.511+0000 7ffacffff640 1 -- 172.21.15.92:0/4198071450 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffadc058ba0 con 0x7ffae80ce380 2024-04-22T07:05:40.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.713+0000 7ffaedaeb640 1 -- 172.21.15.92:0/4198071450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffae80c3e50 con 0x7ffae80ce380 2024-04-22T07:05:40.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.714+0000 7ffacffff640 1 -- 172.21.15.92:0/4198071450 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7ffadc05cea0 con 0x7ffae80ce380 2024-04-22T07:05:40.712 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-04-22T07:05:40.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.716+0000 7ffaedaeb640 1 -- 172.21.15.92:0/4198071450 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ffad406c8f0 msgr2=0x7ffad406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.716+0000 7ffaedaeb640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ffad406c8f0 0x7ffad406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffad8036000 tx=0x7ffad803d040 comp rx=0 tx=0).stop 2024-04-22T07:05:40.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.716+0000 7ffaedaeb640 1 -- 172.21.15.92:0/4198071450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae80ce380 msgr2=0x7ffae80cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.716+0000 7ffaedaeb640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae80ce380 0x7ffae80cb440 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00c3f0 tx=0x7ffadc00c8b0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.716+0000 7ffaedaeb640 1 -- 172.21.15.92:0/4198071450 shutdown_connections 2024-04-22T07:05:40.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.716+0000 7ffaedaeb640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ffad406c8f0 0x7ffad406edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.716+0000 7ffaedaeb640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffae8161350 0x7ffae815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.716+0000 7ffaedaeb640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffae80ce380 0x7ffae80cb440 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.716+0000 7ffaedaeb640 1 --2- 172.21.15.92:0/4198071450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffae8070db0 0x7ffae80c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.716+0000 7ffaedaeb640 1 -- 172.21.15.92:0/4198071450 >> 172.21.15.92:0/4198071450 conn(0x7ffae806db80 msgr2=0x7ffae8157050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:40.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.716+0000 7ffaedaeb640 1 -- 172.21.15.92:0/4198071450 shutdown_connections 2024-04-22T07:05:40.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.716+0000 7ffaedaeb640 1 -- 172.21.15.92:0/4198071450 wait complete. 2024-04-22T07:05:40.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:05:40.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.841+0000 7f0a92e11640 1 Processor -- start 2024-04-22T07:05:40.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.841+0000 7f0a92e11640 1 -- start start 2024-04-22T07:05:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.841+0000 7f0a92e11640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0a8c159360 0x7f0a8c0c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.841+0000 7f0a92e11640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0a8c0c6600 0x7f0a8c0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.842+0000 7f0a92e11640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a8c0c72f0 con 0x7f0a8c162dd0 2024-04-22T07:05:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.842+0000 7f0a92e11640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f0a8c0c6ae0 con 0x7f0a8c159360 2024-04-22T07:05:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.842+0000 7f0a92e11640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f0a8c0c6d30 con 0x7f0a8c0c6600 2024-04-22T07:05:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.842+0000 7f0a90bad640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0a8c159360 0x7f0a8c0c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.842+0000 7f0a90bad640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0a8c159360 0x7f0a8c0c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32990/0 (socket says 172.21.15.92:32990) 2024-04-22T07:05:40.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.842+0000 7f0a90bad640 1 -- 172.21.15.92:0/1138957493 learned_addr learned my addr 172.21.15.92:0/1138957493 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:40.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.842+0000 7f0a8bfff640 1 --2- 172.21.15.92:0/1138957493 >> v2:172.21.15.145:3300/0 conn(0x7f0a8c0c6600 0x7f0a8c0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.842+0000 7f0a90bad640 1 -- 172.21.15.92:0/1138957493 >> v2:172.21.15.145:3300/0 conn(0x7f0a8c0c6600 msgr2=0x7f0a8c0cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:40.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.842+0000 7f0a90bad640 1 --2- 172.21.15.92:0/1138957493 >> v2:172.21.15.145:3300/0 conn(0x7f0a8c0c6600 0x7f0a8c0cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.842+0000 7f0a90bad640 1 -- 172.21.15.92:0/1138957493 >> v1:172.21.15.145:6789/0 conn(0x7f0a8c162dd0 legacy=0x7f0a8c0b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.842+0000 7f0a90bad640 1 -- 172.21.15.92:0/1138957493 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c0c7010 con 0x7f0a8c159360 2024-04-22T07:05:40.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.843+0000 7f0a90bad640 1 --2- 172.21.15.92:0/1138957493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8c159360 0x7f0a8c0c58e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0caa70 tx=0x7f0a8002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700ec1a8715eef65 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:40.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.843+0000 7f0a8b7fe640 1 -- 172.21.15.92:0/1138957493 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8002f850 con 0x7f0a8c159360 2024-04-22T07:05:40.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.843+0000 7f0a8b7fe640 1 -- 172.21.15.92:0/1138957493 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8002f9f0 con 0x7f0a8c159360 2024-04-22T07:05:40.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.843+0000 7f0a8b7fe640 1 -- 172.21.15.92:0/1138957493 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8002fdc0 con 0x7f0a8c159360 2024-04-22T07:05:40.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.844+0000 7f0a92e11640 1 -- 172.21.15.92:0/1138957493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8c159360 msgr2=0x7f0a8c0c58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.844+0000 7f0a92e11640 1 --2- 172.21.15.92:0/1138957493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8c159360 0x7f0a8c0c58e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0caa70 tx=0x7f0a8002ba30 comp rx=0 tx=0).stop 2024-04-22T07:05:40.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.844+0000 7f0a92e11640 1 -- 172.21.15.92:0/1138957493 shutdown_connections 2024-04-22T07:05:40.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.844+0000 7f0a92e11640 1 --2- 172.21.15.92:0/1138957493 >> v2:172.21.15.145:3300/0 conn(0x7f0a8c0c6600 0x7f0a8c0cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.844+0000 7f0a92e11640 1 --2- 172.21.15.92:0/1138957493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8c159360 0x7f0a8c0c58e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.844+0000 7f0a92e11640 1 -- 172.21.15.92:0/1138957493 >> 172.21.15.92:0/1138957493 conn(0x7f0a8c073ca0 msgr2=0x7f0a8c06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:40.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.844+0000 7f0a92e11640 1 -- 172.21.15.92:0/1138957493 shutdown_connections 2024-04-22T07:05:40.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.844+0000 7f0a92e11640 1 -- 172.21.15.92:0/1138957493 wait complete. 2024-04-22T07:05:40.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.846+0000 7f0a92e11640 1 Processor -- start 2024-04-22T07:05:40.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.846+0000 7f0a92e11640 1 -- start start 2024-04-22T07:05:40.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.847+0000 7f0a92e11640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8c0c6600 0x7f0a8c0c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.847+0000 7f0a92e11640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a8c159360 0x7f0a8c15b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.847+0000 7f0a92e11640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8c162dd0 0x7f0a8c15ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.847+0000 7f0a92e11640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c0b5e50 con 0x7f0a8c162dd0 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.847+0000 7f0a92e11640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1628b0 con 0x7f0a8c0c6600 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.847+0000 7f0a92e11640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8c0b5fd0 con 0x7f0a8c159360 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a90bad640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8c0c6600 0x7f0a8c0c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a913ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8c162dd0 0x7f0a8c15ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a90bad640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8c0c6600 0x7f0a8c0c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.92:46038/0 (socket says 172.21.15.92:46038) 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a90bad640 1 -- 172.21.15.92:0/2524313447 learned_addr learned my addr 172.21.15.92:0/2524313447 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a8bfff640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a8c159360 0x7f0a8c15b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a913ae640 1 -- 172.21.15.92:0/2524313447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a8c159360 msgr2=0x7f0a8c15b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a913ae640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a8c159360 0x7f0a8c15b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a913ae640 1 -- 172.21.15.92:0/2524313447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8c0c6600 msgr2=0x7f0a8c0c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a913ae640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8c0c6600 0x7f0a8c0c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a913ae640 1 -- 172.21.15.92:0/2524313447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c1f5330 con 0x7f0a8c162dd0 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a913ae640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8c162dd0 0x7f0a8c15ded0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400c760 tx=0x7f0a8400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:40.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a897fa640 1 -- 172.21.15.92:0/2524313447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84016020 con 0x7f0a8c162dd0 2024-04-22T07:05:40.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a897fa640 1 -- 172.21.15.92:0/2524313447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a84006d50 con 0x7f0a8c162dd0 2024-04-22T07:05:40.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a897fa640 1 -- 172.21.15.92:0/2524313447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84003bb0 con 0x7f0a8c162dd0 2024-04-22T07:05:40.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.848+0000 7f0a92e11640 1 -- 172.21.15.92:0/2524313447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8c0ca1a0 con 0x7f0a8c162dd0 2024-04-22T07:05:40.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.849+0000 7f0a92e11640 1 -- 172.21.15.92:0/2524313447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8c1f58d0 con 0x7f0a8c162dd0 2024-04-22T07:05:40.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.850+0000 7f0a897fa640 1 -- 172.21.15.92:0/2524313447 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a840050d0 con 0x7f0a8c162dd0 2024-04-22T07:05:40.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.850+0000 7f0a92e11640 1 -- 172.21.15.92:0/2524313447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a8c069930 con 0x7f0a8c162dd0 2024-04-22T07:05:40.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.851+0000 7f0a897fa640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0a7006c920 0x7f0a7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.851+0000 7f0a90bad640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0a7006c920 0x7f0a7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.851+0000 7f0a897fa640 1 -- 172.21.15.92:0/2524313447 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f0a8408b640 con 0x7f0a8c162dd0 2024-04-22T07:05:40.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.851+0000 7f0a90bad640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0a7006c920 0x7f0a7006ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009280 tx=0x7f0a80008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:40.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:40.857+0000 7f0a897fa640 1 -- 172.21.15.92:0/2524313447 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a84020020 con 0x7f0a8c162dd0 2024-04-22T07:05:41.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.038+0000 7f0a92e11640 1 -- 172.21.15.92:0/2524313447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0a8c0ce9e0 con 0x7f0a8c162dd0 2024-04-22T07:05:41.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.039+0000 7f0a897fa640 1 -- 172.21.15.92:0/2524313447 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f0a840592c0 con 0x7f0a8c162dd0 2024-04-22T07:05:41.037 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-04-22T07:05:41.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.040+0000 7f0a92e11640 1 -- 172.21.15.92:0/2524313447 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0a7006c920 msgr2=0x7f0a7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.040+0000 7f0a92e11640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0a7006c920 0x7f0a7006ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009280 tx=0x7f0a80008030 comp rx=0 tx=0).stop 2024-04-22T07:05:41.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.040+0000 7f0a92e11640 1 -- 172.21.15.92:0/2524313447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8c162dd0 msgr2=0x7f0a8c15ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.040+0000 7f0a92e11640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8c162dd0 0x7f0a8c15ded0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400c760 tx=0x7f0a8400cc20 comp rx=0 tx=0).stop 2024-04-22T07:05:41.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.041+0000 7f0a92e11640 1 -- 172.21.15.92:0/2524313447 shutdown_connections 2024-04-22T07:05:41.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.041+0000 7f0a92e11640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f0a7006c920 0x7f0a7006ede0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.041+0000 7f0a92e11640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a8c162dd0 0x7f0a8c15ded0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.041+0000 7f0a92e11640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a8c159360 0x7f0a8c15b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.041+0000 7f0a92e11640 1 --2- 172.21.15.92:0/2524313447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8c0c6600 0x7f0a8c0c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.041+0000 7f0a92e11640 1 -- 172.21.15.92:0/2524313447 >> 172.21.15.92:0/2524313447 conn(0x7f0a8c073ca0 msgr2=0x7f0a8c160520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:41.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.041+0000 7f0a92e11640 1 -- 172.21.15.92:0/2524313447 shutdown_connections 2024-04-22T07:05:41.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.041+0000 7f0a92e11640 1 -- 172.21.15.92:0/2524313447 wait complete. 2024-04-22T07:05:41.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:05:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.171+0000 7f9bcabf4640 1 Processor -- start 2024-04-22T07:05:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.171+0000 7f9bcabf4640 1 -- start start 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.172+0000 7f9bcabf4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc4154f30 0x7f9bc415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.172+0000 7f9bcabf4640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9bc415ee40 0x7f9bc40ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.172+0000 7f9bcabf4640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc4155310 con 0x7f9bc415ea00 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.172+0000 7f9bcabf4640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f9bc415b230 con 0x7f9bc415ee40 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.172+0000 7f9bcabf4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc415b510 con 0x7f9bc4154f30 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.172+0000 7f9bc9191640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9bc415ee40 0x7f9bc40ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.172+0000 7f9bc9191640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9bc415ee40 0x7f9bc40ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33012/0 (socket says 172.21.15.92:33012) 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.172+0000 7f9bc9191640 1 -- 172.21.15.92:0/1910981657 learned_addr learned my addr 172.21.15.92:0/1910981657 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.173+0000 7f9bc8990640 1 --2- 172.21.15.92:0/1910981657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc4154f30 0x7f9bc415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.173+0000 7f9bc9191640 1 -- 172.21.15.92:0/1910981657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc4154f30 msgr2=0x7f9bc415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.173+0000 7f9bc9191640 1 --2- 172.21.15.92:0/1910981657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc4154f30 0x7f9bc415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.173+0000 7f9bc9191640 1 -- 172.21.15.92:0/1910981657 >> v1:172.21.15.145:6789/0 conn(0x7f9bc415ea00 legacy=0x7f9bc40cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.173+0000 7f9bc9191640 1 -- 172.21.15.92:0/1910981657 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc415b7f0 con 0x7f9bc415ee40 2024-04-22T07:05:41.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.174+0000 7f9bc9191640 1 --2- 172.21.15.92:0/1910981657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bc415ee40 0x7f9bc40ccea0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009e50 tx=0x7f9bbc02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3841444da8d2658d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:41.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.174+0000 7f9bc37fe640 1 -- 172.21.15.92:0/1910981657 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc02cb90 con 0x7f9bc415ee40 2024-04-22T07:05:41.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.174+0000 7f9bc37fe640 1 -- 172.21.15.92:0/1910981657 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bbc02cd30 con 0x7f9bc415ee40 2024-04-22T07:05:41.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.174+0000 7f9bc37fe640 1 -- 172.21.15.92:0/1910981657 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc034610 con 0x7f9bc415ee40 2024-04-22T07:05:41.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.175+0000 7f9bcabf4640 1 -- 172.21.15.92:0/1910981657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bc415ee40 msgr2=0x7f9bc40ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.175+0000 7f9bcabf4640 1 --2- 172.21.15.92:0/1910981657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bc415ee40 0x7f9bc40ccea0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009e50 tx=0x7f9bbc02bff0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.175+0000 7f9bcabf4640 1 -- 172.21.15.92:0/1910981657 shutdown_connections 2024-04-22T07:05:41.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.175+0000 7f9bcabf4640 1 --2- 172.21.15.92:0/1910981657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bc415ee40 0x7f9bc40ccea0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.175+0000 7f9bcabf4640 1 --2- 172.21.15.92:0/1910981657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc4154f30 0x7f9bc415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.175+0000 7f9bcabf4640 1 -- 172.21.15.92:0/1910981657 >> 172.21.15.92:0/1910981657 conn(0x7f9bc4073ca0 msgr2=0x7f9bc406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:41.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.175+0000 7f9bcabf4640 1 -- 172.21.15.92:0/1910981657 shutdown_connections 2024-04-22T07:05:41.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.175+0000 7f9bcabf4640 1 -- 172.21.15.92:0/1910981657 wait complete. 2024-04-22T07:05:41.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.176+0000 7f9bcabf4640 1 Processor -- start 2024-04-22T07:05:41.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.176+0000 7f9bcabf4640 1 -- start start 2024-04-22T07:05:41.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bcabf4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bc4154f30 0x7f9bc4155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bcabf4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc415ea00 0x7f9bc4146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bcabf4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9bc415ee40 0x7f9bc41560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bcabf4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc415e4e0 con 0x7f9bc4154f30 2024-04-22T07:05:41.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bcabf4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc415e200 con 0x7f9bc415ee40 2024-04-22T07:05:41.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bcabf4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc40c58a0 con 0x7f9bc415ea00 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bc8990640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bc4154f30 0x7f9bc4155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bc8990640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bc4154f30 0x7f9bc4155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33022/0 (socket says 172.21.15.92:33022) 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bc8990640 1 -- 172.21.15.92:0/2293102292 learned_addr learned my addr 172.21.15.92:0/2293102292 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bc9191640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9bc415ee40 0x7f9bc41560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bc3fff640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc415ea00 0x7f9bc4146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bc8990640 1 -- 172.21.15.92:0/2293102292 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc415ea00 msgr2=0x7f9bc4146440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bc8990640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc415ea00 0x7f9bc4146440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bc8990640 1 -- 172.21.15.92:0/2293102292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9bc415ee40 msgr2=0x7f9bc41560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bc8990640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9bc415ee40 0x7f9bc41560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bc8990640 1 -- 172.21.15.92:0/2293102292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc4144af0 con 0x7f9bc4154f30 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.178+0000 7f9bc8990640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bc4154f30 0x7f9bc4155540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800e8a0 tx=0x7f9bb800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.179+0000 7f9bc17fa640 1 -- 172.21.15.92:0/2293102292 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8017070 con 0x7f9bc4154f30 2024-04-22T07:05:41.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.179+0000 7f9bc17fa640 1 -- 172.21.15.92:0/2293102292 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb800c940 con 0x7f9bc4154f30 2024-04-22T07:05:41.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.179+0000 7f9bcabf4640 1 -- 172.21.15.92:0/2293102292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc40d0f70 con 0x7f9bc4154f30 2024-04-22T07:05:41.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.179+0000 7f9bc17fa640 1 -- 172.21.15.92:0/2293102292 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb800cd30 con 0x7f9bc4154f30 2024-04-22T07:05:41.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.179+0000 7f9bcabf4640 1 -- 172.21.15.92:0/2293102292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc40d0560 con 0x7f9bc4154f30 2024-04-22T07:05:41.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.180+0000 7f9bcabf4640 1 -- 172.21.15.92:0/2293102292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc40c5370 con 0x7f9bc4154f30 2024-04-22T07:05:41.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.181+0000 7f9bc17fa640 1 -- 172.21.15.92:0/2293102292 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bb801a070 con 0x7f9bc4154f30 2024-04-22T07:05:41.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.181+0000 7f9bc17fa640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9ba806c8f0 0x7f9ba806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.182+0000 7f9bc17fa640 1 -- 172.21.15.92:0/2293102292 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f9bb808b970 con 0x7f9bc4154f30 2024-04-22T07:05:41.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.182+0000 7f9bc3fff640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9ba806c8f0 0x7f9ba806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.182+0000 7f9bc3fff640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9ba806c8f0 0x7f9ba806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9bc41477e0 tx=0x7f9bb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:41.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.186+0000 7f9bc17fa640 1 -- 172.21.15.92:0/2293102292 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bc40c5370 con 0x7f9bc4154f30 2024-04-22T07:05:41.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.386+0000 7f9bcabf4640 1 -- 172.21.15.92:0/2293102292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9bc4069930 con 0x7f9bc4154f30 2024-04-22T07:05:41.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.386+0000 7f9bc17fa640 1 -- 172.21.15.92:0/2293102292 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f9bb8059710 con 0x7f9bc4154f30 2024-04-22T07:05:41.384 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-04-22T07:05:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.388+0000 7f9bcabf4640 1 -- 172.21.15.92:0/2293102292 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9ba806c8f0 msgr2=0x7f9ba806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.388+0000 7f9bcabf4640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9ba806c8f0 0x7f9ba806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9bc41477e0 tx=0x7f9bb0009040 comp rx=0 tx=0).stop 2024-04-22T07:05:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.388+0000 7f9bcabf4640 1 -- 172.21.15.92:0/2293102292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bc4154f30 msgr2=0x7f9bc4155540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.389+0000 7f9bcabf4640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bc4154f30 0x7f9bc4155540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800e8a0 tx=0x7f9bb800ed60 comp rx=0 tx=0).stop 2024-04-22T07:05:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.389+0000 7f9bcabf4640 1 -- 172.21.15.92:0/2293102292 shutdown_connections 2024-04-22T07:05:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.389+0000 7f9bcabf4640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f9ba806c8f0 0x7f9ba806edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.389+0000 7f9bcabf4640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9bc415ee40 0x7f9bc41560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.389+0000 7f9bcabf4640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc415ea00 0x7f9bc4146440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.389+0000 7f9bcabf4640 1 --2- 172.21.15.92:0/2293102292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bc4154f30 0x7f9bc4155540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.389+0000 7f9bcabf4640 1 -- 172.21.15.92:0/2293102292 >> 172.21.15.92:0/2293102292 conn(0x7f9bc4073ca0 msgr2=0x7f9bc41485f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:41.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.389+0000 7f9bcabf4640 1 -- 172.21.15.92:0/2293102292 shutdown_connections 2024-04-22T07:05:41.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.389+0000 7f9bcabf4640 1 -- 172.21.15.92:0/2293102292 wait complete. 2024-04-22T07:05:41.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T07:05:41.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.498+0000 7fc15fbce640 1 Processor -- start 2024-04-22T07:05:41.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.498+0000 7fc15fbce640 1 -- start start 2024-04-22T07:05:41.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.499+0000 7fc15fbce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc158154fa0 0x7fc15815b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.499+0000 7fc15fbce640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc15815e4a0 con 0x7fc1580cb8c0 2024-04-22T07:05:41.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.499+0000 7fc15fbce640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc15815e780 con 0x7fc1580cb480 2024-04-22T07:05:41.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.499+0000 7fc15fbce640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fc15815ea60 con 0x7fc158154fa0 2024-04-22T07:05:41.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.499+0000 7fc15d96a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc158154fa0 0x7fc15815b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.499+0000 7fc15d96a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc158154fa0 0x7fc15815b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33034/0 (socket says 172.21.15.92:33034) 2024-04-22T07:05:41.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.499+0000 7fc15d96a640 1 -- 172.21.15.92:0/3185153434 learned_addr learned my addr 172.21.15.92:0/3185153434 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:41.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.499+0000 7fc15e16b640 1 --1- 172.21.15.92:0/3185153434 >> v1:172.21.15.92:6789/0 conn(0x7fc1580cb8c0 0x7fc1580b4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50844/0 (socket says 172.21.15.92:50844) 2024-04-22T07:05:41.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.500+0000 7fc15d96a640 1 -- 172.21.15.92:0/3185153434 >> v1:172.21.15.145:6789/0 conn(0x7fc1580cb480 legacy=0x7fc1580d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.500+0000 7fc15d96a640 1 -- 172.21.15.92:0/3185153434 >> v1:172.21.15.92:6789/0 conn(0x7fc1580cb8c0 legacy=0x7fc1580b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.500+0000 7fc15c968640 1 -- 172.21.15.92:0/3185153434 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 908866447 0 0) 0x7fc15815e4a0 con 0x7fc1580cb8c0 2024-04-22T07:05:41.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.500+0000 7fc15d96a640 1 -- 172.21.15.92:0/3185153434 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15815ed40 con 0x7fc158154fa0 2024-04-22T07:05:41.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.501+0000 7fc15d96a640 1 --2- 172.21.15.92:0/3185153434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc158154fa0 0x7fc15815b830 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc154009870 tx=0x7fc15402bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74f8ad6f10d947e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:41.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.501+0000 7fc15c968640 1 -- 172.21.15.92:0/3185153434 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15402d870 con 0x7fc158154fa0 2024-04-22T07:05:41.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.501+0000 7fc15c968640 1 -- 172.21.15.92:0/3185153434 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc15402da10 con 0x7fc158154fa0 2024-04-22T07:05:41.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.501+0000 7fc15c968640 1 -- 172.21.15.92:0/3185153434 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15402dde0 con 0x7fc158154fa0 2024-04-22T07:05:41.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.501+0000 7fc15fbce640 1 -- 172.21.15.92:0/3185153434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc158154fa0 msgr2=0x7fc15815b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.501+0000 7fc15fbce640 1 --2- 172.21.15.92:0/3185153434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc158154fa0 0x7fc15815b830 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc154009870 tx=0x7fc15402bab0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.501+0000 7fc15fbce640 1 -- 172.21.15.92:0/3185153434 shutdown_connections 2024-04-22T07:05:41.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.501+0000 7fc15fbce640 1 --2- 172.21.15.92:0/3185153434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc158154fa0 0x7fc15815b830 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.501+0000 7fc15fbce640 1 -- 172.21.15.92:0/3185153434 >> 172.21.15.92:0/3185153434 conn(0x7fc158073930 msgr2=0x7fc15806cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:41.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.502+0000 7fc15fbce640 1 -- 172.21.15.92:0/3185153434 shutdown_connections 2024-04-22T07:05:41.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.502+0000 7fc15fbce640 1 -- 172.21.15.92:0/3185153434 wait complete. 2024-04-22T07:05:41.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.503+0000 7fc15fbce640 1 Processor -- start 2024-04-22T07:05:41.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.503+0000 7fc15fbce640 1 -- start start 2024-04-22T07:05:41.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.504+0000 7fc15fbce640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1580cb480 0x7fc1581472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.504+0000 7fc15fbce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1580cb8c0 0x7fc15815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.504+0000 7fc15fbce640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc158154fa0 0x7fc158148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.504+0000 7fc15fbce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc158155380 con 0x7fc1580cb8c0 2024-04-22T07:05:41.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.504+0000 7fc15fbce640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc1580d0ba0 con 0x7fc158154fa0 2024-04-22T07:05:41.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.504+0000 7fc15fbce640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc158152a20 con 0x7fc1580cb480 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.504+0000 7fc15d169640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1580cb8c0 0x7fc15815cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.504+0000 7fc15d169640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1580cb8c0 0x7fc15815cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33044/0 (socket says 172.21.15.92:33044) 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.504+0000 7fc15d169640 1 -- 172.21.15.92:0/1012260517 learned_addr learned my addr 172.21.15.92:0/1012260517 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.505+0000 7fc15e16b640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc158154fa0 0x7fc158148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.505+0000 7fc15d96a640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1580cb480 0x7fc1581472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.505+0000 7fc15d169640 1 -- 172.21.15.92:0/1012260517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1580cb480 msgr2=0x7fc1581472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.505+0000 7fc15d169640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1580cb480 0x7fc1581472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.505+0000 7fc15d169640 1 -- 172.21.15.92:0/1012260517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc158154fa0 msgr2=0x7fc158148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.505+0000 7fc15d169640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc158154fa0 0x7fc158148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.505+0000 7fc15d169640 1 -- 172.21.15.92:0/1012260517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15815be80 con 0x7fc1580cb8c0 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.505+0000 7fc15e16b640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc158154fa0 0x7fc158148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.505+0000 7fc15d169640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1580cb8c0 0x7fc15815cab0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc14800c3f0 tx=0x7fc14800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.506+0000 7fc14e7fc640 1 -- 172.21.15.92:0/1012260517 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148017070 con 0x7fc1580cb8c0 2024-04-22T07:05:41.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.506+0000 7fc14e7fc640 1 -- 172.21.15.92:0/1012260517 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1480052f0 con 0x7fc1580cb8c0 2024-04-22T07:05:41.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.506+0000 7fc14e7fc640 1 -- 172.21.15.92:0/1012260517 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148003830 con 0x7fc1580cb8c0 2024-04-22T07:05:41.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.506+0000 7fc15fbce640 1 -- 172.21.15.92:0/1012260517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15815e050 con 0x7fc1580cb8c0 2024-04-22T07:05:41.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.506+0000 7fc15fbce640 1 -- 172.21.15.92:0/1012260517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1581526b0 con 0x7fc1580cb8c0 2024-04-22T07:05:41.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.507+0000 7fc15fbce640 1 -- 172.21.15.92:0/1012260517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc15815c360 con 0x7fc1580cb8c0 2024-04-22T07:05:41.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.508+0000 7fc14e7fc640 1 -- 172.21.15.92:0/1012260517 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc14801a070 con 0x7fc1580cb8c0 2024-04-22T07:05:41.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.509+0000 7fc14e7fc640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc13c06c940 0x7fc13c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.509+0000 7fc15d96a640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc13c06c940 0x7fc13c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.509+0000 7fc14e7fc640 1 -- 172.21.15.92:0/1012260517 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fc14808aed0 con 0x7fc1580cb8c0 2024-04-22T07:05:41.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.509+0000 7fc15d96a640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc13c06c940 0x7fc13c06ee00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc154036000 tx=0x7fc154008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:41.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.513+0000 7fc14e7fc640 1 -- 172.21.15.92:0/1012260517 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc15815c360 con 0x7fc1580cb8c0 2024-04-22T07:05:41.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.801+0000 7fc15fbce640 1 -- 172.21.15.92:0/1012260517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc158069930 con 0x7fc1580cb8c0 2024-04-22T07:05:41.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.801+0000 7fc14e7fc640 1 -- 172.21.15.92:0/1012260517 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v23) ==== 80+0+124 (secure 0 0 0) 0x7fc148059280 con 0x7fc1580cb8c0 2024-04-22T07:05:41.800 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.145:6836/1058270486 2024-04-23T07:05:35.574446+0000 2024-04-22T07:05:41.800 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.145:6833/1058270486 2024-04-23T07:05:35.574446+0000 2024-04-22T07:05:41.800 INFO:teuthology.orchestra.run.smithi092.stderr:listed 2 entries 2024-04-22T07:05:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.803+0000 7fc15fbce640 1 -- 172.21.15.92:0/1012260517 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc13c06c940 msgr2=0x7fc13c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.803+0000 7fc15fbce640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc13c06c940 0x7fc13c06ee00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc154036000 tx=0x7fc154008030 comp rx=0 tx=0).stop 2024-04-22T07:05:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.803+0000 7fc15fbce640 1 -- 172.21.15.92:0/1012260517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1580cb8c0 msgr2=0x7fc15815cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.803+0000 7fc15fbce640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1580cb8c0 0x7fc15815cab0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc14800c3f0 tx=0x7fc14800c8b0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.803+0000 7fc15fbce640 1 -- 172.21.15.92:0/1012260517 shutdown_connections 2024-04-22T07:05:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.803+0000 7fc15fbce640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7fc13c06c940 0x7fc13c06ee00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.803+0000 7fc15fbce640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc158154fa0 0x7fc158148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.803+0000 7fc15fbce640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1580cb8c0 0x7fc15815cab0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.803+0000 7fc15fbce640 1 --2- 172.21.15.92:0/1012260517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1580cb480 0x7fc1581472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.803+0000 7fc15fbce640 1 -- 172.21.15.92:0/1012260517 >> 172.21.15.92:0/1012260517 conn(0x7fc158073930 msgr2=0x7fc1580b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.803+0000 7fc15fbce640 1 -- 172.21.15.92:0/1012260517 shutdown_connections 2024-04-22T07:05:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.803+0000 7fc15fbce640 1 -- 172.21.15.92:0/1012260517 wait complete. 2024-04-22T07:05:41.812 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:05:41.812 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T07:05:41.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:05:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.983+0000 7f07ef967640 1 Processor -- start 2024-04-22T07:05:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.983+0000 7f07ef967640 1 -- start start 2024-04-22T07:05:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ef967640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f07e80d0b70 0x7f07e8154a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ef967640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f07e80d0f50 0x7f07e8154f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ef967640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07e80cb120 0x7f07e815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ef967640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f07e815ba00 con 0x7f07e80d0f50 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ef967640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f07e815ab30 con 0x7f07e80d0b70 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ef967640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f07e815adf0 con 0x7f07e80cb120 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ecf02640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f07e80d0f50 0x7f07e8154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ecf02640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f07e80d0f50 0x7f07e8154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33056/0 (socket says 172.21.15.92:33056) 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ecf02640 1 -- 172.21.15.92:0/1906913249 learned_addr learned my addr 172.21.15.92:0/1906913249 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ed703640 1 --2- 172.21.15.92:0/1906913249 >> v2:172.21.15.145:3300/0 conn(0x7f07e80d0b70 0x7f07e8154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07edf04640 1 --2- 172.21.15.92:0/1906913249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07e80cb120 0x7f07e815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ecf02640 1 -- 172.21.15.92:0/1906913249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07e80cb120 msgr2=0x7f07e815e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ecf02640 1 --2- 172.21.15.92:0/1906913249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07e80cb120 0x7f07e815e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ecf02640 1 -- 172.21.15.92:0/1906913249 >> v2:172.21.15.145:3300/0 conn(0x7f07e80d0b70 msgr2=0x7f07e8154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ecf02640 1 --2- 172.21.15.92:0/1906913249 >> v2:172.21.15.145:3300/0 conn(0x7f07e80d0b70 0x7f07e8154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07ecf02640 1 -- 172.21.15.92:0/1906913249 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e815b0d0 con 0x7f07e80d0f50 2024-04-22T07:05:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.984+0000 7f07edf04640 1 --2- 172.21.15.92:0/1906913249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07e80cb120 0x7f07e815e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.985+0000 7f07ecf02640 1 --2- 172.21.15.92:0/1906913249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e80d0f50 0x7f07e8154f40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f07e80d14d0 tx=0x7f07d802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cd13f29480986d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.986+0000 7f07dffff640 1 -- 172.21.15.92:0/1906913249 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d802e070 con 0x7f07e80d0f50 2024-04-22T07:05:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.986+0000 7f07dffff640 1 -- 172.21.15.92:0/1906913249 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d8037070 con 0x7f07e80d0f50 2024-04-22T07:05:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.986+0000 7f07dffff640 1 -- 172.21.15.92:0/1906913249 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8031a00 con 0x7f07e80d0f50 2024-04-22T07:05:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.986+0000 7f07ef967640 1 -- 172.21.15.92:0/1906913249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e80d0f50 msgr2=0x7f07e8154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.986+0000 7f07ef967640 1 --2- 172.21.15.92:0/1906913249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e80d0f50 0x7f07e8154f40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f07e80d14d0 tx=0x7f07d802b880 comp rx=0 tx=0).stop 2024-04-22T07:05:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.986+0000 7f07ef967640 1 -- 172.21.15.92:0/1906913249 shutdown_connections 2024-04-22T07:05:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.986+0000 7f07ef967640 1 --2- 172.21.15.92:0/1906913249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07e80cb120 0x7f07e815e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.986+0000 7f07ef967640 1 --2- 172.21.15.92:0/1906913249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e80d0f50 0x7f07e8154f40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.986+0000 7f07ef967640 1 --2- 172.21.15.92:0/1906913249 >> v2:172.21.15.145:3300/0 conn(0x7f07e80d0b70 0x7f07e8154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.986+0000 7f07ef967640 1 -- 172.21.15.92:0/1906913249 >> 172.21.15.92:0/1906913249 conn(0x7f07e80736b0 msgr2=0x7f07e8073ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.986+0000 7f07ef967640 1 -- 172.21.15.92:0/1906913249 shutdown_connections 2024-04-22T07:05:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.987+0000 7f07ef967640 1 -- 172.21.15.92:0/1906913249 wait complete. 2024-04-22T07:05:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.988+0000 7f07ef967640 1 Processor -- start 2024-04-22T07:05:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.988+0000 7f07ef967640 1 -- start start 2024-04-22T07:05:41.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.989+0000 7f07ef967640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07e80cb120 0x7f07e8146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.989+0000 7f07ef967640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e80d0b70 0x7f07e80cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.989+0000 7f07ecf02640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e80d0b70 0x7f07e80cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.989+0000 7f07ecf02640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e80d0b70 0x7f07e80cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33068/0 (socket says 172.21.15.92:33068) 2024-04-22T07:05:41.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.989+0000 7f07ef967640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07e80d0f50 0x7f07e8152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.989+0000 7f07ecf02640 1 -- 172.21.15.92:0/3525752957 learned_addr learned my addr 172.21.15.92:0/3525752957 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:41.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.989+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f07e80cae00 con 0x7f07e80d0b70 2024-04-22T07:05:41.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.989+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f07e815b690 con 0x7f07e80d0f50 2024-04-22T07:05:41.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.989+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f07e80caf80 con 0x7f07e80cb120 2024-04-22T07:05:41.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.990+0000 7f07ed703640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07e80cb120 0x7f07e8146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.990+0000 7f07edf04640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07e80d0f50 0x7f07e8152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.990+0000 7f07ecf02640 1 -- 172.21.15.92:0/3525752957 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07e80cb120 msgr2=0x7f07e8146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.990+0000 7f07ecf02640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07e80cb120 0x7f07e8146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.990+0000 7f07ecf02640 1 -- 172.21.15.92:0/3525752957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07e80d0f50 msgr2=0x7f07e8152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.990+0000 7f07ecf02640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07e80d0f50 0x7f07e8152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.990+0000 7f07ecf02640 1 -- 172.21.15.92:0/3525752957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e815db60 con 0x7f07e80d0b70 2024-04-22T07:05:41.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.990+0000 7f07ecf02640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e80d0b70 0x7f07e80cd340 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f07d802b8c0 tx=0x7f07d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:41.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.990+0000 7f07ddffb640 1 -- 172.21.15.92:0/3525752957 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8032880 con 0x7f07e80d0b70 2024-04-22T07:05:41.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.990+0000 7f07ddffb640 1 -- 172.21.15.92:0/3525752957 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d8032a40 con 0x7f07e80d0b70 2024-04-22T07:05:41.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.990+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e815de20 con 0x7f07e80d0b70 2024-04-22T07:05:41.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.990+0000 7f07ddffb640 1 -- 172.21.15.92:0/3525752957 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8032e30 con 0x7f07e80d0b70 2024-04-22T07:05:41.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.991+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e81f1000 con 0x7f07e80d0b70 2024-04-22T07:05:41.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.992+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e8069930 con 0x7f07e80d0b70 2024-04-22T07:05:41.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.992+0000 7f07ddffb640 1 -- 172.21.15.92:0/3525752957 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07d80453f0 con 0x7f07e80d0b70 2024-04-22T07:05:41.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.993+0000 7f07ddffb640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f07cc06c8f0 0x7f07cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.993+0000 7f07ddffb640 1 -- 172.21.15.92:0/3525752957 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f07d80ada20 con 0x7f07e80d0b70 2024-04-22T07:05:41.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.993+0000 7f07ed703640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f07cc06c8f0 0x7f07cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.993+0000 7f07ed703640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f07cc06c8f0 0x7f07cc06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f07e4007920 tx=0x7f07e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:41.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:41.997+0000 7f07ddffb640 1 -- 172.21.15.92:0/3525752957 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07d8040070 con 0x7f07e80d0b70 2024-04-22T07:05:42.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.245+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07e80b34c0 con 0x7f07e80d0b70 2024-04-22T07:05:42.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.246+0000 7f07ddffb640 1 -- 172.21.15.92:0/3525752957 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16881 (secure 0 0 0) 0x7f07d804ee20 con 0x7f07e80d0b70 2024-04-22T07:05:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-22T07:05:42.245 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"72eb29c5-e441-491c-9309-4fb4ef16ce27","created":"2024-04-22T07:03:45.780510+0000","modified":"2024-04-22T07:05:35.574514+0000","last_up_change":"2024-04-22T07:03:56.750732+0000","last_in_change":"2024-04-22T07:03:48.098217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:03:57.947601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:01.058111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:02.085602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d554be68-0854-4df8-b5a3-2d07f668274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6825","nonce":3408935700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6827","nonce":3408935700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6831","nonce":3408935700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3408935700},{"type":"v1","addr":"172.21.15.92:6829","nonce":3408935700}]},"public_addr":"172.21.15.92:6825/3408935700","cluster_addr":"172.21.15.92:6827/3408935700","heartbeat_back_addr":"172.21.15.92:6831/3408935700","heartbeat_front_addr":"172.21.15.92:6829/3408935700","state":["exists","up"]},{"osd":1,"uuid":"963d3334-a43c-4aaf-b6b3-841b520d6c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6817","nonce":2805470585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6819","nonce":2805470585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6823","nonce":2805470585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2805470585},{"type":"v1","addr":"172.21.15.92:6821","nonce":2805470585}]},"public_addr":"172.21.15.92:6817/2805470585","cluster_addr":"172.21.15.92:6819/2805470585","heartbeat_back_addr":"172.21.15.92:6823/2805470585","heartbeat_front_addr":"172.21.15.92:6821/2805470585","state":["exists","up"]},{"osd":2,"uuid":"480ebbfe-df23-4a47-98b4-f46d6b4f4bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6809","nonce":3594806740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6811","nonce":3594806740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6815","nonce":3594806740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3594806740},{"type":"v1","addr":"172.21.15.92:6813","nonce":3594806740}]},"public_addr":"172.21.15.92:6809/3594806740","cluster_addr":"172.21.15.92:6811/3594806740","heartbeat_back_addr":"172.21.15.92:6815/3594806740","heartbeat_front_addr":"172.21.15.92:6813/3594806740","state":["exists","up"]},{"osd":3,"uuid":"3edb4281-5113-4f74-930f-bf39cfae1d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6801","nonce":24674040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6803","nonce":24674040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6807","nonce":24674040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":24674040},{"type":"v1","addr":"172.21.15.92:6805","nonce":24674040}]},"public_addr":"172.21.15.92:6801/24674040","cluster_addr":"172.21.15.92:6803/24674040","heartbeat_back_addr":"172.21.15.92:6807/24674040","heartbeat_front_addr":"172.21.15.92:6805/24674040","state":["exists","up"]},{"osd":4,"uuid":"078f7b2e-f718-498e-82c2-907970d4497e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6806","nonce":3070790698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6810","nonce":3070790698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6815","nonce":3070790698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3070790698},{"type":"v1","addr":"172.21.15.145:6813","nonce":3070790698}]},"public_addr":"172.21.15.145:6806/3070790698","cluster_addr":"172.21.15.145:6810/3070790698","heartbeat_back_addr":"172.21.15.145:6815/3070790698","heartbeat_front_addr":"172.21.15.145:6813/3070790698","state":["exists","up"]},{"osd":5,"uuid":"fe0e1c02-ccf8-441a-8ed6-afcff9db1565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6801","nonce":2981429006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6803","nonce":2981429006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6809","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6811","nonce":2981429006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6805","nonce":2981429006},{"type":"v1","addr":"172.21.15.145:6807","nonce":2981429006}]},"public_addr":"172.21.15.145:6801/2981429006","cluster_addr":"172.21.15.145:6803/2981429006","heartbeat_back_addr":"172.21.15.145:6811/2981429006","heartbeat_front_addr":"172.21.15.145:6807/2981429006","state":["exists","up"]},{"osd":6,"uuid":"7c9dc8bd-f1c0-4005-956d-928a1ce7636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6817","nonce":290698449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6819","nonce":290698449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6823","nonce":290698449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":290698449},{"type":"v1","addr":"172.21.15.145:6821","nonce":290698449}]},"public_addr":"172.21.15.145:6817/290698449","cluster_addr":"172.21.15.145:6819/290698449","heartbeat_back_addr":"172.21.15.145:6823/290698449","heartbeat_front_addr":"172.21.15.145:6821/290698449","state":["exists","up"]},{"osd":7,"uuid":"51065971-9bc8-4005-baf2-39a4506048a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6825","nonce":97436481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6827","nonce":97436481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6831","nonce":97436481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":97436481},{"type":"v1","addr":"172.21.15.145:6829","nonce":97436481}]},"public_addr":"172.21.15.145:6825/97436481","cluster_addr":"172.21.15.145:6827/97436481","heartbeat_back_addr":"172.21.15.145:6831/97436481","heartbeat_front_addr":"172.21.15.145:6829/97436481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.232313+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.297782+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.206197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.205683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.257938+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.183293+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.217149+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:53.261571+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:6836/1058270486":"2024-04-23T07:05:35.574446+0000","172.21.15.145:6833/1058270486":"2024-04-23T07:05:35.574446+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:05:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.249+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f07cc06c8f0 msgr2=0x7f07cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.249+0000 7f07ef967640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f07cc06c8f0 0x7f07cc06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f07e4007920 tx=0x7f07e4008040 comp rx=0 tx=0).stop 2024-04-22T07:05:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.249+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e80d0b70 msgr2=0x7f07e80cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.249+0000 7f07ef967640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e80d0b70 0x7f07e80cd340 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f07d802b8c0 tx=0x7f07d80046c0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.250+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 shutdown_connections 2024-04-22T07:05:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.250+0000 7f07ef967640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f07cc06c8f0 0x7f07cc06edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.250+0000 7f07ef967640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07e80d0f50 0x7f07e8152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.250+0000 7f07ef967640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e80d0b70 0x7f07e80cd340 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.250+0000 7f07ef967640 1 --2- 172.21.15.92:0/3525752957 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07e80cb120 0x7f07e8146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.250+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 >> 172.21.15.92:0/3525752957 conn(0x7f07e80736b0 msgr2=0x7f07e815e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.250+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 shutdown_connections 2024-04-22T07:05:42.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.250+0000 7f07ef967640 1 -- 172.21.15.92:0/3525752957 wait complete. 2024-04-22T07:05:42.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:03:57.947601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:01.058111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:02.085602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:05:42.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:05:42.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.391+0000 7f89e20f2640 1 Processor -- start 2024-04-22T07:05:42.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.391+0000 7f89e20f2640 1 -- start start 2024-04-22T07:05:42.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.391+0000 7f89e20f2640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89dc154f30 0x7f89dc15ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.391+0000 7f89e20f2640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f89dc15ea00 0x7f89dc0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89e20f2640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f89dc15ee40 0x7f89dc0cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89e20f2640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f89dc155310 con 0x7f89dc15ea00 2024-04-22T07:05:42.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89e20f2640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f89dc15b230 con 0x7f89dc15ee40 2024-04-22T07:05:42.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89e20f2640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f89dc15b510 con 0x7f89dc154f30 2024-04-22T07:05:42.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89daffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f89dc15ea00 0x7f89dc0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89daffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f89dc15ea00 0x7f89dc0cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33070/0 (socket says 172.21.15.92:33070) 2024-04-22T07:05:42.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89daffd640 1 -- 172.21.15.92:0/2047583909 learned_addr learned my addr 172.21.15.92:0/2047583909 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:42.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89dbfff640 1 --2- 172.21.15.92:0/2047583909 >> v2:172.21.15.145:3300/0 conn(0x7f89dc15ee40 0x7f89dc0cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89db7fe640 1 --2- 172.21.15.92:0/2047583909 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89dc154f30 0x7f89dc15ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89daffd640 1 -- 172.21.15.92:0/2047583909 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89dc154f30 msgr2=0x7f89dc15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:42.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89daffd640 1 --2- 172.21.15.92:0/2047583909 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89dc154f30 0x7f89dc15ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89daffd640 1 -- 172.21.15.92:0/2047583909 >> v2:172.21.15.145:3300/0 conn(0x7f89dc15ee40 msgr2=0x7f89dc0cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:42.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89daffd640 1 --2- 172.21.15.92:0/2047583909 >> v2:172.21.15.145:3300/0 conn(0x7f89dc15ee40 0x7f89dc0cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.392+0000 7f89daffd640 1 -- 172.21.15.92:0/2047583909 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc15b7f0 con 0x7f89dc15ea00 2024-04-22T07:05:42.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.393+0000 7f89daffd640 1 --2- 172.21.15.92:0/2047583909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15ea00 0x7f89dc0cbf20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0b3730 tx=0x7f89d002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7490f077dcf4aa80 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:42.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.393+0000 7f89da7fc640 1 -- 172.21.15.92:0/2047583909 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0034070 con 0x7f89dc15ea00 2024-04-22T07:05:42.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.393+0000 7f89da7fc640 1 -- 172.21.15.92:0/2047583909 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d0030860 con 0x7f89dc15ea00 2024-04-22T07:05:42.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.393+0000 7f89da7fc640 1 -- 172.21.15.92:0/2047583909 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0030c30 con 0x7f89dc15ea00 2024-04-22T07:05:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.393+0000 7f89e20f2640 1 -- 172.21.15.92:0/2047583909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15ea00 msgr2=0x7f89dc0cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.393+0000 7f89e20f2640 1 --2- 172.21.15.92:0/2047583909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15ea00 0x7f89dc0cbf20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0b3730 tx=0x7f89d002b9a0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.394+0000 7f89e20f2640 1 -- 172.21.15.92:0/2047583909 shutdown_connections 2024-04-22T07:05:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.394+0000 7f89e20f2640 1 --2- 172.21.15.92:0/2047583909 >> v2:172.21.15.145:3300/0 conn(0x7f89dc15ee40 0x7f89dc0cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.394+0000 7f89e20f2640 1 --2- 172.21.15.92:0/2047583909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15ea00 0x7f89dc0cbf20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.394+0000 7f89e20f2640 1 --2- 172.21.15.92:0/2047583909 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89dc154f30 0x7f89dc15ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.394+0000 7f89e20f2640 1 -- 172.21.15.92:0/2047583909 >> 172.21.15.92:0/2047583909 conn(0x7f89dc073ca0 msgr2=0x7f89dc06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.394+0000 7f89e20f2640 1 -- 172.21.15.92:0/2047583909 shutdown_connections 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.394+0000 7f89e20f2640 1 -- 172.21.15.92:0/2047583909 wait complete. 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.395+0000 7f89e20f2640 1 Processor -- start 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.395+0000 7f89e20f2640 1 -- start start 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89e20f2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89dc154f30 0x7f89dc0d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89e20f2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15ea00 0x7f89dc0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89e20f2640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89dc15ee40 0x7f89dc15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89e20f2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc15e4e0 con 0x7f89dc15ea00 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89e20f2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc15e200 con 0x7f89dc154f30 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89e20f2640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f89dc0c58a0 con 0x7f89dc15ee40 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89daffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15ea00 0x7f89dc0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89daffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15ea00 0x7f89dc0ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33086/0 (socket says 172.21.15.92:33086) 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89daffd640 1 -- 172.21.15.92:0/1605627712 learned_addr learned my addr 172.21.15.92:0/1605627712 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89db7fe640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89dc154f30 0x7f89dc0d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89dbfff640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89dc15ee40 0x7f89dc15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89daffd640 1 -- 172.21.15.92:0/1605627712 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89dc15ee40 msgr2=0x7f89dc15d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89daffd640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89dc15ee40 0x7f89dc15d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89daffd640 1 -- 172.21.15.92:0/1605627712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89dc154f30 msgr2=0x7f89dc0d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89daffd640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89dc154f30 0x7f89dc0d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89daffd640 1 -- 172.21.15.92:0/1605627712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc0d0230 con 0x7f89dc15ea00 2024-04-22T07:05:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89daffd640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15ea00 0x7f89dc0ce390 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f89d0000c00 tx=0x7f89d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.396+0000 7f89c3fff640 1 -- 172.21.15.92:0/1605627712 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0034040 con 0x7f89dc15ea00 2024-04-22T07:05:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.397+0000 7f89c3fff640 1 -- 172.21.15.92:0/1605627712 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d0039d90 con 0x7f89dc15ea00 2024-04-22T07:05:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.397+0000 7f89e20f2640 1 -- 172.21.15.92:0/1605627712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89dc157f30 con 0x7f89dc15ea00 2024-04-22T07:05:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.397+0000 7f89e20f2640 1 -- 172.21.15.92:0/1605627712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89dc0d04d0 con 0x7f89dc15ea00 2024-04-22T07:05:42.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.397+0000 7f89c3fff640 1 -- 172.21.15.92:0/1605627712 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0043630 con 0x7f89dc15ea00 2024-04-22T07:05:42.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.398+0000 7f89e20f2640 1 -- 172.21.15.92:0/1605627712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89dc069930 con 0x7f89dc15ea00 2024-04-22T07:05:42.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.398+0000 7f89c3fff640 1 -- 172.21.15.92:0/1605627712 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89d004a060 con 0x7f89dc15ea00 2024-04-22T07:05:42.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.399+0000 7f89c3fff640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f89c806c920 0x7f89c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.399+0000 7f89c3fff640 1 -- 172.21.15.92:0/1605627712 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f89d00addd0 con 0x7f89dc15ea00 2024-04-22T07:05:42.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.399+0000 7f89db7fe640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f89c806c920 0x7f89c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.399+0000 7f89db7fe640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f89c806c920 0x7f89c806ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f89cc006fd0 tx=0x7f89cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:42.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.403+0000 7f89c3fff640 1 -- 172.21.15.92:0/1605627712 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89d003e070 con 0x7f89dc15ea00 2024-04-22T07:05:42.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.586+0000 7f89e20f2640 1 -- 172.21.15.92:0/1605627712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f89dc0cbd00 con 0x7f89dc15ea00 2024-04-22T07:05:42.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.586+0000 7f89c3fff640 1 -- 172.21.15.92:0/1605627712 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f89d0037360 con 0x7f89dc15ea00 2024-04-22T07:05:42.584 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-04-22T07:05:42.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.588+0000 7f89e20f2640 1 -- 172.21.15.92:0/1605627712 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f89c806c920 msgr2=0x7f89c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.588+0000 7f89e20f2640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f89c806c920 0x7f89c806ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f89cc006fd0 tx=0x7f89cc008040 comp rx=0 tx=0).stop 2024-04-22T07:05:42.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.588+0000 7f89e20f2640 1 -- 172.21.15.92:0/1605627712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15ea00 msgr2=0x7f89dc0ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.588+0000 7f89e20f2640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15ea00 0x7f89dc0ce390 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f89d0000c00 tx=0x7f89d00046c0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.588+0000 7f89e20f2640 1 -- 172.21.15.92:0/1605627712 shutdown_connections 2024-04-22T07:05:42.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.588+0000 7f89e20f2640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f89c806c920 0x7f89c806ede0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.588+0000 7f89e20f2640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89dc15ee40 0x7f89dc15d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.588+0000 7f89e20f2640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89dc15ea00 0x7f89dc0ce390 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.588+0000 7f89e20f2640 1 --2- 172.21.15.92:0/1605627712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89dc154f30 0x7f89dc0d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.588+0000 7f89e20f2640 1 -- 172.21.15.92:0/1605627712 >> 172.21.15.92:0/1605627712 conn(0x7f89dc073ca0 msgr2=0x7f89dc0c5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:42.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.588+0000 7f89e20f2640 1 -- 172.21.15.92:0/1605627712 shutdown_connections 2024-04-22T07:05:42.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.588+0000 7f89e20f2640 1 -- 172.21.15.92:0/1605627712 wait complete. 2024-04-22T07:05:42.601 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:05:42.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.703+0000 7f4e3a270640 1 Processor -- start 2024-04-22T07:05:42.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.703+0000 7f4e3a270640 1 -- start start 2024-04-22T07:05:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.703+0000 7f4e3a270640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e3415ac10 0x7f4e340cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.703+0000 7f4e3a270640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e340cc460 0x7f4e340ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.704+0000 7f4e3a270640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e34155310 con 0x7f4e34154f30 2024-04-22T07:05:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.704+0000 7f4e3a270640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f4e3415b230 con 0x7f4e340cc460 2024-04-22T07:05:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.704+0000 7f4e3a270640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3415b510 con 0x7f4e3415ac10 2024-04-22T07:05:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.704+0000 7f4e3880d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e340cc460 0x7f4e340ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.704+0000 7f4e3880d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e340cc460 0x7f4e340ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33088/0 (socket says 172.21.15.92:33088) 2024-04-22T07:05:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.704+0000 7f4e3880d640 1 -- 172.21.15.92:0/1367717636 learned_addr learned my addr 172.21.15.92:0/1367717636 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.704+0000 7f4e337fe640 1 --2- 172.21.15.92:0/1367717636 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e3415ac10 0x7f4e340cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.704+0000 7f4e3880d640 1 -- 172.21.15.92:0/1367717636 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e3415ac10 msgr2=0x7f4e340cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.704+0000 7f4e3880d640 1 --2- 172.21.15.92:0/1367717636 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e3415ac10 0x7f4e340cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.704+0000 7f4e3880d640 1 -- 172.21.15.92:0/1367717636 >> v1:172.21.15.145:6789/0 conn(0x7f4e34154f30 legacy=0x7f4e3415ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.704+0000 7f4e3880d640 1 -- 172.21.15.92:0/1367717636 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3415b7f0 con 0x7f4e340cc460 2024-04-22T07:05:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.705+0000 7f4e3880d640 1 --2- 172.21.15.92:0/1367717636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340cc460 0x7f4e340ccea0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009e50 tx=0x7f4e2c02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2038d33ce485a956 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.705+0000 7f4e32ffd640 1 -- 172.21.15.92:0/1367717636 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c02cb90 con 0x7f4e340cc460 2024-04-22T07:05:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.705+0000 7f4e32ffd640 1 -- 172.21.15.92:0/1367717636 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e2c02cd30 con 0x7f4e340cc460 2024-04-22T07:05:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.705+0000 7f4e32ffd640 1 -- 172.21.15.92:0/1367717636 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c034610 con 0x7f4e340cc460 2024-04-22T07:05:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.705+0000 7f4e3a270640 1 -- 172.21.15.92:0/1367717636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340cc460 msgr2=0x7f4e340ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.705+0000 7f4e3a270640 1 --2- 172.21.15.92:0/1367717636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340cc460 0x7f4e340ccea0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009e50 tx=0x7f4e2c02bff0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.706+0000 7f4e3a270640 1 -- 172.21.15.92:0/1367717636 shutdown_connections 2024-04-22T07:05:42.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.706+0000 7f4e3a270640 1 --2- 172.21.15.92:0/1367717636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340cc460 0x7f4e340ccea0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.706+0000 7f4e3a270640 1 --2- 172.21.15.92:0/1367717636 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e3415ac10 0x7f4e340cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.706+0000 7f4e3a270640 1 -- 172.21.15.92:0/1367717636 >> 172.21.15.92:0/1367717636 conn(0x7f4e34073ca0 msgr2=0x7f4e3406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:42.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.706+0000 7f4e3a270640 1 -- 172.21.15.92:0/1367717636 shutdown_connections 2024-04-22T07:05:42.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.706+0000 7f4e3a270640 1 -- 172.21.15.92:0/1367717636 wait complete. 2024-04-22T07:05:42.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.707+0000 7f4e3a270640 1 Processor -- start 2024-04-22T07:05:42.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.707+0000 7f4e3a270640 1 -- start start 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e3a270640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340cc460 0x7f4e34149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e3a270640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e34154f30 0x7f4e3414f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e3a270640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e3415ac10 0x7f4e34153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e33fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340cc460 0x7f4e34149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e3a270640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3415e4e0 con 0x7f4e340cc460 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e3a270640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3415e200 con 0x7f4e3415ac10 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e3a270640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3415f110 con 0x7f4e34154f30 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e33fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340cc460 0x7f4e34149100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33102/0 (socket says 172.21.15.92:33102) 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e33fff640 1 -- 172.21.15.92:0/3623941675 learned_addr learned my addr 172.21.15.92:0/3623941675 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e337fe640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e34154f30 0x7f4e3414f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e3880d640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e3415ac10 0x7f4e34153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e33fff640 1 -- 172.21.15.92:0/3623941675 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e34154f30 msgr2=0x7f4e3414f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e33fff640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e34154f30 0x7f4e3414f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e33fff640 1 -- 172.21.15.92:0/3623941675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e3415ac10 msgr2=0x7f4e34153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e33fff640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e3415ac10 0x7f4e34153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.708+0000 7f4e33fff640 1 -- 172.21.15.92:0/3623941675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e341f3ab0 con 0x7f4e340cc460 2024-04-22T07:05:42.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.709+0000 7f4e33fff640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340cc460 0x7f4e34149100 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400c430 tx=0x7f4e2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:42.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.709+0000 7f4e30ff9640 1 -- 172.21.15.92:0/3623941675 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24017070 con 0x7f4e340cc460 2024-04-22T07:05:42.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.709+0000 7f4e30ff9640 1 -- 172.21.15.92:0/3623941675 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e240052d0 con 0x7f4e340cc460 2024-04-22T07:05:42.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.709+0000 7f4e3a270640 1 -- 172.21.15.92:0/3623941675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e34152be0 con 0x7f4e340cc460 2024-04-22T07:05:42.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.709+0000 7f4e30ff9640 1 -- 172.21.15.92:0/3623941675 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24003830 con 0x7f4e340cc460 2024-04-22T07:05:42.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.709+0000 7f4e3a270640 1 -- 172.21.15.92:0/3623941675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e341531a0 con 0x7f4e340cc460 2024-04-22T07:05:42.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.710+0000 7f4e3a270640 1 -- 172.21.15.92:0/3623941675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e34069930 con 0x7f4e340cc460 2024-04-22T07:05:42.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.711+0000 7f4e30ff9640 1 -- 172.21.15.92:0/3623941675 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e2401a070 con 0x7f4e340cc460 2024-04-22T07:05:42.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.712+0000 7f4e30ff9640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4e2006c8f0 0x7f4e2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:42.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.712+0000 7f4e30ff9640 1 -- 172.21.15.92:0/3623941675 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f4e2408aed0 con 0x7f4e340cc460 2024-04-22T07:05:42.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.712+0000 7f4e337fe640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4e2006c8f0 0x7f4e2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:42.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.713+0000 7f4e337fe640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4e2006c8f0 0x7f4e2006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4e340caee0 tx=0x7f4e28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:42.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.716+0000 7f4e30ff9640 1 -- 172.21.15.92:0/3623941675 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e240592c0 con 0x7f4e340cc460 2024-04-22T07:05:42.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.943+0000 7f4e3a270640 1 -- 172.21.15.92:0/3623941675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4e340cbd00 con 0x7f4e340cc460 2024-04-22T07:05:42.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.943+0000 7f4e30ff9640 1 -- 172.21.15.92:0/3623941675 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f4e24058bc0 con 0x7f4e340cc460 2024-04-22T07:05:42.941 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-04-22T07:05:42.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.945+0000 7f4e3a270640 1 -- 172.21.15.92:0/3623941675 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4e2006c8f0 msgr2=0x7f4e2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.945+0000 7f4e3a270640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4e2006c8f0 0x7f4e2006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4e340caee0 tx=0x7f4e28008040 comp rx=0 tx=0).stop 2024-04-22T07:05:42.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.945+0000 7f4e3a270640 1 -- 172.21.15.92:0/3623941675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340cc460 msgr2=0x7f4e34149100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:42.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.945+0000 7f4e3a270640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340cc460 0x7f4e34149100 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400c430 tx=0x7f4e2400c8f0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.945+0000 7f4e3a270640 1 -- 172.21.15.92:0/3623941675 shutdown_connections 2024-04-22T07:05:42.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.945+0000 7f4e3a270640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f4e2006c8f0 0x7f4e2006edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.945+0000 7f4e3a270640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e3415ac10 0x7f4e34153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.945+0000 7f4e3a270640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4e34154f30 0x7f4e3414f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.945+0000 7f4e3a270640 1 --2- 172.21.15.92:0/3623941675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340cc460 0x7f4e34149100 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:42.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.945+0000 7f4e3a270640 1 -- 172.21.15.92:0/3623941675 >> 172.21.15.92:0/3623941675 conn(0x7f4e34073ca0 msgr2=0x7f4e3415a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:42.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.945+0000 7f4e3a270640 1 -- 172.21.15.92:0/3623941675 shutdown_connections 2024-04-22T07:05:42.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:42.945+0000 7f4e3a270640 1 -- 172.21.15.92:0/3623941675 wait complete. 2024-04-22T07:05:42.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:05:43.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.056+0000 7f8ce7b0d640 1 Processor -- start 2024-04-22T07:05:43.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.056+0000 7f8ce7b0d640 1 -- start start 2024-04-22T07:05:43.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.057+0000 7f8ce7b0d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ce0154f30 0x7f8ce015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.057+0000 7f8ce7b0d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ce0155310 con 0x7f8ce015ea00 2024-04-22T07:05:43.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.057+0000 7f8ce7b0d640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ce015b230 con 0x7f8ce015ee70 2024-04-22T07:05:43.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.057+0000 7f8ce7b0d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f8ce015b510 con 0x7f8ce0154f30 2024-04-22T07:05:43.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.057+0000 7f8ce58a9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ce0154f30 0x7f8ce015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.057+0000 7f8ce50a8640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8ce015ea00 0x7f8ce00cbf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50858/0 (socket says 172.21.15.92:50858) 2024-04-22T07:05:43.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.057+0000 7f8ce50a8640 1 -- 172.21.15.92:0/426245767 learned_addr learned my addr 172.21.15.92:0/426245767 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:05:43.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.058+0000 7f8ce58a9640 1 -- 172.21.15.92:0/426245767 >> v1:172.21.15.145:6789/0 conn(0x7f8ce015ee70 legacy=0x7f8ce00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.058+0000 7f8ce58a9640 1 -- 172.21.15.92:0/426245767 >> v1:172.21.15.92:6789/0 conn(0x7f8ce015ea00 legacy=0x7f8ce00cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.058+0000 7f8ce58a9640 1 -- 172.21.15.92:0/426245767 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce015b7f0 con 0x7f8ce0154f30 2024-04-22T07:05:43.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.059+0000 7f8ce58a9640 1 --2- 172.21.15.92:0/426245767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce0154f30 0x7f8ce015ac10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0097a0 tx=0x7f8cdc02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbf5b2a58f8e511 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:43.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.059+0000 7f8ce48a7640 1 -- 172.21.15.92:0/426245767 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc02f810 con 0x7f8ce0154f30 2024-04-22T07:05:43.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.059+0000 7f8ce48a7640 1 -- 172.21.15.92:0/426245767 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cdc02f9b0 con 0x7f8ce0154f30 2024-04-22T07:05:43.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.059+0000 7f8ce48a7640 1 -- 172.21.15.92:0/426245767 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc02fd80 con 0x7f8ce0154f30 2024-04-22T07:05:43.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.059+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/426245767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce0154f30 msgr2=0x7f8ce015ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.059+0000 7f8ce7b0d640 1 --2- 172.21.15.92:0/426245767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce0154f30 0x7f8ce015ac10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0097a0 tx=0x7f8cdc02b9f0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.059+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/426245767 shutdown_connections 2024-04-22T07:05:43.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.060+0000 7f8ce7b0d640 1 --2- 172.21.15.92:0/426245767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce0154f30 0x7f8ce015ac10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.060+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/426245767 >> 172.21.15.92:0/426245767 conn(0x7f8ce0073ca0 msgr2=0x7f8ce006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:43.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.060+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/426245767 shutdown_connections 2024-04-22T07:05:43.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.060+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/426245767 wait complete. 2024-04-22T07:05:43.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.061+0000 7f8ce7b0d640 1 Processor -- start 2024-04-22T07:05:43.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.061+0000 7f8ce7b0d640 1 -- start start 2024-04-22T07:05:43.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.061+0000 7f8ce7b0d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8ce0154f30 0x7f8ce01f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.061+0000 7f8ce7b0d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce015ea00 0x7f8ce0159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.061+0000 7f8ce7b0d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ce015ee70 0x7f8ce0155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.061+0000 7f8ce7b0d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce015e4e0 con 0x7f8ce015ea00 2024-04-22T07:05:43.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.061+0000 7f8ce7b0d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce015e200 con 0x7f8ce015ee70 2024-04-22T07:05:43.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.061+0000 7f8ce7b0d640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce00c58a0 con 0x7f8ce0154f30 2024-04-22T07:05:43.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce50a8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce015ea00 0x7f8ce0159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce50a8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce015ea00 0x7f8ce0159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33122/0 (socket says 172.21.15.92:33122) 2024-04-22T07:05:43.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce50a8640 1 -- 172.21.15.92:0/3381582661 learned_addr learned my addr 172.21.15.92:0/3381582661 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:43.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce58a9640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8ce0154f30 0x7f8ce01f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce60aa640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ce015ee70 0x7f8ce0155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce50a8640 1 -- 172.21.15.92:0/3381582661 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8ce0154f30 msgr2=0x7f8ce01f14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce50a8640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8ce0154f30 0x7f8ce01f14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce50a8640 1 -- 172.21.15.92:0/3381582661 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ce015ee70 msgr2=0x7f8ce0155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce50a8640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ce015ee70 0x7f8ce0155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce50a8640 1 -- 172.21.15.92:0/3381582661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce014eb30 con 0x7f8ce015ea00 2024-04-22T07:05:43.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce60aa640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ce015ee70 0x7f8ce0155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:43.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce50a8640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce015ea00 0x7f8ce0159d70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000c4d0 tx=0x7f8cd000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:43.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8cd67fc640 1 -- 172.21.15.92:0/3381582661 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0017070 con 0x7f8ce015ea00 2024-04-22T07:05:43.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8cd67fc640 1 -- 172.21.15.92:0/3381582661 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd0005ce0 con 0x7f8ce015ea00 2024-04-22T07:05:43.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/3381582661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce014edd0 con 0x7f8ce015ea00 2024-04-22T07:05:43.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8cd67fc640 1 -- 172.21.15.92:0/3381582661 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0007270 con 0x7f8ce015ea00 2024-04-22T07:05:43.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.062+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/3381582661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce0159760 con 0x7f8ce015ea00 2024-04-22T07:05:43.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.064+0000 7f8cd67fc640 1 -- 172.21.15.92:0/3381582661 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8cd001a070 con 0x7f8ce015ea00 2024-04-22T07:05:43.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.064+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/3381582661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce00c54b0 con 0x7f8ce015ea00 2024-04-22T07:05:43.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.065+0000 7f8cd67fc640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8cc406c920 0x7f8cc406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.065+0000 7f8cd67fc640 1 -- 172.21.15.92:0/3381582661 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f8cd008af30 con 0x7f8ce015ea00 2024-04-22T07:05:43.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.065+0000 7f8ce58a9640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8cc406c920 0x7f8cc406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.066+0000 7f8ce58a9640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8cc406c920 0x7f8cc406ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc036000 tx=0x7f8cdc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:43.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.069+0000 7f8cd67fc640 1 -- 172.21.15.92:0/3381582661 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ce00c54b0 con 0x7f8ce015ea00 2024-04-22T07:05:43.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.257+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/3381582661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8ce0069930 con 0x7f8ce015ea00 2024-04-22T07:05:43.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.258+0000 7f8cd67fc640 1 -- 172.21.15.92:0/3381582661 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f8cd0058ba0 con 0x7f8ce015ea00 2024-04-22T07:05:43.258 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-04-22T07:05:43.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.260+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/3381582661 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8cc406c920 msgr2=0x7f8cc406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.260+0000 7f8ce7b0d640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8cc406c920 0x7f8cc406ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc036000 tx=0x7f8cdc008030 comp rx=0 tx=0).stop 2024-04-22T07:05:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.260+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/3381582661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce015ea00 msgr2=0x7f8ce0159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.260+0000 7f8ce7b0d640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce015ea00 0x7f8ce0159d70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000c4d0 tx=0x7f8cd000c990 comp rx=0 tx=0).stop 2024-04-22T07:05:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.260+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/3381582661 shutdown_connections 2024-04-22T07:05:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.260+0000 7f8ce7b0d640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7f8cc406c920 0x7f8cc406ede0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.260+0000 7f8ce7b0d640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ce015ee70 0x7f8ce0155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.260+0000 7f8ce7b0d640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce015ea00 0x7f8ce0159d70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.260+0000 7f8ce7b0d640 1 --2- 172.21.15.92:0/3381582661 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8ce0154f30 0x7f8ce01f14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.260+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/3381582661 >> 172.21.15.92:0/3381582661 conn(0x7f8ce0073ca0 msgr2=0x7f8ce0149e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.260+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/3381582661 shutdown_connections 2024-04-22T07:05:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.260+0000 7f8ce7b0d640 1 -- 172.21.15.92:0/3381582661 wait complete. 2024-04-22T07:05:43.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T07:05:43.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.377+0000 7ff06989a640 1 Processor -- start 2024-04-22T07:05:43.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.377+0000 7ff06989a640 1 -- start start 2024-04-22T07:05:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.377+0000 7ff06989a640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff0640b4970 0x7ff0640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.377+0000 7ff06989a640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff06415bc70 con 0x7ff06415b830 2024-04-22T07:05:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.377+0000 7ff06989a640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff06415e4a0 con 0x7ff064154fa0 2024-04-22T07:05:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.377+0000 7ff06989a640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff0640b5360 con 0x7ff0640b4970 2024-04-22T07:05:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff0627fc640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff06415b830 0x7ff0640d0490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50874/0 (socket says 172.21.15.92:50874) 2024-04-22T07:05:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff0627fc640 1 -- 172.21.15.92:0/3795412211 learned_addr learned my addr 172.21.15.92:0/3795412211 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-04-22T07:05:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff0637fe640 1 --2- 172.21.15.92:0/3795412211 >> v2:172.21.15.145:3300/0 conn(0x7ff0640b4970 0x7ff0640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 138264279 0 0) 0x7ff06415bc70 con 0x7ff06415b830 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff04c003520 con 0x7ff06415b830 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1813933408 0 0) 0x7ff06415e4a0 con 0x7ff064154fa0 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff06415bc70 con 0x7ff064154fa0 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3524810353 0 0) 0x7ff04c003520 con 0x7ff06415b830 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 >> v2:172.21.15.145:3300/0 conn(0x7ff0640b4970 msgr2=0x7ff0640b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 --2- 172.21.15.92:0/3795412211 >> v2:172.21.15.145:3300/0 conn(0x7ff0640b4970 0x7ff0640b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 >> v1:172.21.15.145:6789/0 conn(0x7ff064154fa0 legacy=0x7ff0640cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06415e780 con 0x7ff06415b830 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3993173310 0 0) 0x7ff058001670 con 0x7ff06415b830 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 >> v1:172.21.15.92:6789/0 conn(0x7ff06415b830 legacy=0x7ff0640d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 --2- 172.21.15.92:0/3795412211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff04c003b90 0x7ff04c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 --2- 172.21.15.92:0/3795412211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff04c004800 0x7ff04c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 --2- 172.21.15.92:0/3795412211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff04c0051c0 0x7ff04c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff0640b5360 con 0x7ff04c003b90 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c00c0a0 con 0x7ff04c004800 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.378+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff04c00c2a0 con 0x7ff04c0051c0 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.379+0000 7ff062ffd640 1 --2- 172.21.15.92:0/3795412211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff04c003b90 0x7ff04c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.379+0000 7ff0627fc640 1 --2- 172.21.15.92:0/3795412211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff04c004800 0x7ff04c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.379+0000 7ff0637fe640 1 --2- 172.21.15.92:0/3795412211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff04c0051c0 0x7ff04c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.379+0000 7ff062ffd640 1 -- 172.21.15.92:0/3795412211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff04c0051c0 msgr2=0x7ff04c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.379+0000 7ff062ffd640 1 --2- 172.21.15.92:0/3795412211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff04c0051c0 0x7ff04c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.379+0000 7ff062ffd640 1 -- 172.21.15.92:0/3795412211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff04c004800 msgr2=0x7ff04c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.379+0000 7ff062ffd640 1 --2- 172.21.15.92:0/3795412211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff04c004800 0x7ff04c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.379+0000 7ff062ffd640 1 -- 172.21.15.92:0/3795412211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04c003520 con 0x7ff04c003b90 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.379+0000 7ff0637fe640 1 -- 172.21.15.92:0/3795412211 reap_dead start 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.380+0000 7ff062ffd640 1 --2- 172.21.15.92:0/3795412211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff04c003b90 0x7ff04c003fb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff054001840 tx=0x7ff04c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11d005b69946459 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.380+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054037070 con 0x7ff04c003b90 2024-04-22T07:05:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.380+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff054032620 con 0x7ff04c003b90 2024-04-22T07:05:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.380+0000 7ff061ffb640 1 -- 172.21.15.92:0/3795412211 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054001460 con 0x7ff04c003b90 2024-04-22T07:05:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.380+0000 7ff06989a640 1 -- 172.21.15.92:0/3795412211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff04c003b90 msgr2=0x7ff04c003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.380+0000 7ff06989a640 1 --2- 172.21.15.92:0/3795412211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff04c003b90 0x7ff04c003fb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff054001840 tx=0x7ff04c005830 comp rx=0 tx=0).stop 2024-04-22T07:05:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.381+0000 7ff06989a640 1 -- 172.21.15.92:0/3795412211 shutdown_connections 2024-04-22T07:05:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.381+0000 7ff06989a640 1 --2- 172.21.15.92:0/3795412211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff04c003b90 0x7ff04c003fb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.381+0000 7ff06989a640 1 -- 172.21.15.92:0/3795412211 >> 172.21.15.92:0/3795412211 conn(0x7ff064073930 msgr2=0x7ff06406cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.381+0000 7ff06989a640 1 -- 172.21.15.92:0/3795412211 shutdown_connections 2024-04-22T07:05:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.381+0000 7ff06989a640 1 -- 172.21.15.92:0/3795412211 wait complete. 2024-04-22T07:05:43.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.382+0000 7ff06989a640 1 Processor -- start 2024-04-22T07:05:43.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.383+0000 7ff06989a640 1 -- start start 2024-04-22T07:05:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff06989a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff04c003b90 0x7ff0641f1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff06989a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff06414ec30 0x7ff0640d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff06989a640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0640d0fd0 0x7ff06414d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff06989a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff06415f020 con 0x7ff06414ec30 2024-04-22T07:05:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff06989a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff06415ed40 con 0x7ff04c003b90 2024-04-22T07:05:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff06989a640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff064155380 con 0x7ff0640d0fd0 2024-04-22T07:05:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff0627fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff06414ec30 0x7ff0640d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff0627fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff06414ec30 0x7ff0640d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33134/0 (socket says 172.21.15.92:33134) 2024-04-22T07:05:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff0627fc640 1 -- 172.21.15.92:0/781284951 learned_addr learned my addr 172.21.15.92:0/781284951 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-04-22T07:05:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff062ffd640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff04c003b90 0x7ff0641f1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff0637fe640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0640d0fd0 0x7ff06414d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff0627fc640 1 -- 172.21.15.92:0/781284951 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0640d0fd0 msgr2=0x7ff06414d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff0627fc640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0640d0fd0 0x7ff06414d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff0627fc640 1 -- 172.21.15.92:0/781284951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff04c003b90 msgr2=0x7ff0641f1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff0627fc640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff04c003b90 0x7ff0641f1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff0627fc640 1 -- 172.21.15.92:0/781284951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06414c980 con 0x7ff06414ec30 2024-04-22T07:05:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff0627fc640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff06414ec30 0x7ff0640d0a90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff058004ec0 tx=0x7ff058007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.384+0000 7ff04bfff640 1 -- 172.21.15.92:0/781284951 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05800f7b0 con 0x7ff06414ec30 2024-04-22T07:05:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.385+0000 7ff04bfff640 1 -- 172.21.15.92:0/781284951 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff058018070 con 0x7ff06414ec30 2024-04-22T07:05:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.385+0000 7ff06989a640 1 -- 172.21.15.92:0/781284951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06414ba90 con 0x7ff06414ec30 2024-04-22T07:05:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.385+0000 7ff04bfff640 1 -- 172.21.15.92:0/781284951 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05800fbb0 con 0x7ff06414ec30 2024-04-22T07:05:43.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.385+0000 7ff06989a640 1 -- 172.21.15.92:0/781284951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06414c050 con 0x7ff06414ec30 2024-04-22T07:05:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.386+0000 7ff06989a640 1 -- 172.21.15.92:0/781284951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff064069930 con 0x7ff06414ec30 2024-04-22T07:05:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.386+0000 7ff04bfff640 1 -- 172.21.15.92:0/781284951 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff05800fd50 con 0x7ff06414ec30 2024-04-22T07:05:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.387+0000 7ff04bfff640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff05006c8f0 0x7ff05006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.387+0000 7ff04bfff640 1 -- 172.21.15.92:0/781284951 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7ff05808bbb0 con 0x7ff06414ec30 2024-04-22T07:05:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.387+0000 7ff062ffd640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff05006c8f0 0x7ff05006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.388+0000 7ff062ffd640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff05006c8f0 0x7ff05006edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff054001810 tx=0x7ff054009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:43.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.391+0000 7ff04bfff640 1 -- 172.21.15.92:0/781284951 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff058059f10 con 0x7ff06414ec30 2024-04-22T07:05:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.591+0000 7ff06989a640 1 -- 172.21.15.92:0/781284951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff06414d3f0 con 0x7ff06414ec30 2024-04-22T07:05:43.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.591+0000 7ff04bfff640 1 -- 172.21.15.92:0/781284951 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v23) ==== 80+0+124 (secure 0 0 0) 0x7ff058059810 con 0x7ff06414ec30 2024-04-22T07:05:43.589 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.145:6836/1058270486 2024-04-23T07:05:35.574446+0000 2024-04-22T07:05:43.590 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.145:6833/1058270486 2024-04-23T07:05:35.574446+0000 2024-04-22T07:05:43.590 INFO:teuthology.orchestra.run.smithi092.stderr:listed 2 entries 2024-04-22T07:05:43.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.593+0000 7ff06989a640 1 -- 172.21.15.92:0/781284951 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff05006c8f0 msgr2=0x7ff05006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.593+0000 7ff06989a640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff05006c8f0 0x7ff05006edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff054001810 tx=0x7ff054009030 comp rx=0 tx=0).stop 2024-04-22T07:05:43.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.593+0000 7ff06989a640 1 -- 172.21.15.92:0/781284951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff06414ec30 msgr2=0x7ff0640d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.593+0000 7ff06989a640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff06414ec30 0x7ff0640d0a90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff058004ec0 tx=0x7ff058007430 comp rx=0 tx=0).stop 2024-04-22T07:05:43.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.594+0000 7ff06989a640 1 -- 172.21.15.92:0/781284951 shutdown_connections 2024-04-22T07:05:43.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.594+0000 7ff06989a640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.92:6832/479317737,v1:172.21.15.92:6833/479317737] conn(0x7ff05006c8f0 0x7ff05006edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.594+0000 7ff06989a640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0640d0fd0 0x7ff06414d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.594+0000 7ff06989a640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff06414ec30 0x7ff0640d0a90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.594+0000 7ff06989a640 1 --2- 172.21.15.92:0/781284951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff04c003b90 0x7ff0641f1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.594+0000 7ff06989a640 1 -- 172.21.15.92:0/781284951 >> 172.21.15.92:0/781284951 conn(0x7ff064073930 msgr2=0x7ff0640c6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:43.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.594+0000 7ff06989a640 1 -- 172.21.15.92:0/781284951 shutdown_connections 2024-04-22T07:05:43.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:05:43.594+0000 7ff06989a640 1 -- 172.21.15.92:0/781284951 wait complete. 2024-04-22T07:05:43.605 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2024-04-22T07:05:43.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:05:43.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-22T07:05:43.630 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:05:43.630 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2024-04-22T07:05:43.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:05:43.631 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-22T07:05:45.206 INFO:tasks.ceph.osd.0:Sent signal 15 2024-04-22T07:05:45.207 INFO:tasks.ceph.osd.1:Sent signal 15 2024-04-22T07:05:45.207 INFO:tasks.ceph.osd.2:Sent signal 15 2024-04-22T07:05:45.207 INFO:tasks.ceph.osd.3:Sent signal 15 2024-04-22T07:05:45.208 INFO:tasks.ceph.osd.4:Sent signal 15 2024-04-22T07:05:45.208 INFO:tasks.ceph.osd.5:Sent signal 15 2024-04-22T07:05:45.208 INFO:tasks.ceph.osd.6:Sent signal 15 2024-04-22T07:05:45.208 INFO:tasks.ceph.osd.7:Sent signal 15 2024-04-22T07:05:45.208 INFO:tasks.ceph.mds.e:Sent signal 15 2024-04-22T07:05:45.209 INFO:tasks.ceph.mds.d:Sent signal 15 2024-04-22T07:05:45.209 INFO:tasks.ceph.mon.a:Sent signal 15 2024-04-22T07:05:45.209 INFO:tasks.ceph.mon.b:Sent signal 15 2024-04-22T07:05:45.209 INFO:tasks.ceph.mon.c:Sent signal 15 2024-04-22T07:05:45.209 INFO:tasks.ceph.mgr.y:Sent signal 15 2024-04-22T07:05:45.210 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-04-22T07:05:45.210 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-22T07:05:45.209+0000 7f2388cf5640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33407) UID: 0 2024-04-22T07:05:45.210 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-22T07:05:45.209+0000 7f2388cf5640 -1 osd.0 23 *** Got signal Terminated *** 2024-04-22T07:05:45.211 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-04-22T07:05:45.209+0000 7f2388cf5640 -1 osd.0 23 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:05:45.211 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-22T07:05:45.210+0000 7f72df12b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33412) UID: 0 2024-04-22T07:05:45.211 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-22T07:05:45.210+0000 7f72df12b640 -1 osd.1 23 *** Got signal Terminated *** 2024-04-22T07:05:45.211 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-04-22T07:05:45.210+0000 7f72df12b640 -1 osd.1 23 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:05:45.212 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-22T07:05:45.210+0000 7fe9e5af5640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33418) UID: 0 2024-04-22T07:05:45.212 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-22T07:05:45.210+0000 7fe9e5af5640 -1 osd.2 23 *** Got signal Terminated *** 2024-04-22T07:05:45.212 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-04-22T07:05:45.210+0000 7fe9e5af5640 -1 osd.2 23 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:05:45.212 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-22T07:05:45.211+0000 7fc97b9bf640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33424) UID: 0 2024-04-22T07:05:45.213 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-22T07:05:45.211+0000 7fc97b9bf640 -1 osd.3 23 *** Got signal Terminated *** 2024-04-22T07:05:45.213 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-22T07:05:45.211+0000 7fc97b9bf640 -1 osd.3 23 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:05:45.213 INFO:tasks.ceph.mds.e.smithi092.stderr:2024-04-22T07:05:45.211+0000 7ff9534df640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i e (PID: 39275) UID: 0 2024-04-22T07:05:45.213 INFO:tasks.ceph.mds.e.smithi092.stderr:2024-04-22T07:05:45.211+0000 7ff9534df640 -1 mds.e *** got signal Terminated *** 2024-04-22T07:05:45.213 INFO:tasks.ceph.mon.a.smithi092.stderr:2024-04-22T07:05:45.212+0000 7fb9a2f1d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 33130) UID: 0 2024-04-22T07:05:45.213 INFO:tasks.ceph.mon.a.smithi092.stderr:2024-04-22T07:05:45.212+0000 7fb9a2f1d640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-22T07:05:45.214 INFO:tasks.ceph.mgr.y.smithi092.stderr:2024-04-22T07:05:45.212+0000 7fdde4d89640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i y (PID: 33166) UID: 0 2024-04-22T07:05:45.214 INFO:tasks.ceph.osd.4.smithi145.stderr:2024-04-22T07:05:45.211+0000 7ff1e22dc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33675) UID: 0 2024-04-22T07:05:45.214 INFO:tasks.ceph.osd.4.smithi145.stderr:2024-04-22T07:05:45.211+0000 7ff1e22dc640 -1 osd.4 23 *** Got signal Terminated *** 2024-04-22T07:05:45.214 INFO:tasks.ceph.osd.4.smithi145.stderr:2024-04-22T07:05:45.211+0000 7ff1e22dc640 -1 osd.4 23 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:05:45.215 INFO:tasks.ceph.osd.5.smithi145.stderr:2024-04-22T07:05:45.211+0000 7f2ddd4d0640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33676) UID: 0 2024-04-22T07:05:45.215 INFO:tasks.ceph.osd.5.smithi145.stderr:2024-04-22T07:05:45.211+0000 7f2ddd4d0640 -1 osd.5 23 *** Got signal Terminated *** 2024-04-22T07:05:45.215 INFO:tasks.ceph.osd.5.smithi145.stderr:2024-04-22T07:05:45.211+0000 7f2ddd4d0640 -1 osd.5 23 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:05:45.215 INFO:tasks.ceph.osd.6.smithi145.stderr:2024-04-22T07:05:45.211+0000 7f870ee66640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33680) UID: 0 2024-04-22T07:05:45.215 INFO:tasks.ceph.osd.6.smithi145.stderr:2024-04-22T07:05:45.211+0000 7f870ee66640 -1 osd.6 23 *** Got signal Terminated *** 2024-04-22T07:05:45.215 INFO:tasks.ceph.osd.6.smithi145.stderr:2024-04-22T07:05:45.211+0000 7f870ee66640 -1 osd.6 23 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:05:45.216 INFO:tasks.ceph.osd.7.smithi145.stderr:2024-04-22T07:05:45.211+0000 7fb6408de640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33683) UID: 0 2024-04-22T07:05:45.216 INFO:tasks.ceph.osd.7.smithi145.stderr:2024-04-22T07:05:45.211+0000 7fb6408de640 -1 osd.7 23 *** Got signal Terminated *** 2024-04-22T07:05:45.216 INFO:tasks.ceph.osd.7.smithi145.stderr:2024-04-22T07:05:45.211+0000 7fb6408de640 -1 osd.7 23 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:05:45.216 INFO:tasks.ceph.mds.d.smithi145.stderr:2024-04-22T07:05:45.211+0000 7f0a6be10640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 39372) UID: 0 2024-04-22T07:05:45.216 INFO:tasks.ceph.mds.d.smithi145.stderr:2024-04-22T07:05:45.211+0000 7f0a6be10640 -1 mds.d *** got signal Terminated *** 2024-04-22T07:05:45.216 INFO:tasks.ceph.mon.b.smithi145.stderr:2024-04-22T07:05:45.212+0000 7fd8da937640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 33096) UID: 0 2024-04-22T07:05:45.217 INFO:tasks.ceph.mon.b.smithi145.stderr:2024-04-22T07:05:45.212+0000 7fd8da937640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-04-22T07:05:45.217 INFO:tasks.ceph.mon.c.smithi145.stderr:2024-04-22T07:05:45.212+0000 7efbff01c640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 33098) UID: 0 2024-04-22T07:05:45.217 INFO:tasks.ceph.mon.c.smithi145.stderr:2024-04-22T07:05:45.212+0000 7efbff01c640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2024-04-22T07:05:45.411 INFO:tasks.ceph.mgr.x.smithi145.stderr:daemon-helper: command crashed with signal 15 2024-04-22T07:06:05.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:06:05.321 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:06:08.723 INFO:tasks.workunit.client.0.smithi092.stderr:Note: switching to '86f7587a5a09af35f0895e1a2d08527638fae697'. 2024-04-22T07:06:08.723 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-04-22T07:06:08.723 INFO:tasks.workunit.client.0.smithi092.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T07:06:08.723 INFO:tasks.workunit.client.0.smithi092.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T07:06:08.723 INFO:tasks.workunit.client.0.smithi092.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T07:06:08.723 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-04-22T07:06:08.723 INFO:tasks.workunit.client.0.smithi092.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T07:06:08.723 INFO:tasks.workunit.client.0.smithi092.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T07:06:08.723 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-04-22T07:06:08.723 INFO:tasks.workunit.client.0.smithi092.stderr: git switch -c 2024-04-22T07:06:08.724 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-04-22T07:06:08.724 INFO:tasks.workunit.client.0.smithi092.stderr:Or undo this operation with: 2024-04-22T07:06:08.724 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-04-22T07:06:08.724 INFO:tasks.workunit.client.0.smithi092.stderr: git switch - 2024-04-22T07:06:08.724 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-04-22T07:06:08.724 INFO:tasks.workunit.client.0.smithi092.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T07:06:08.724 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-04-22T07:06:08.724 INFO:tasks.workunit.client.0.smithi092.stderr:HEAD is now at 86f7587a5a0 qa/fsx: use the master branch to build the xfstest-dev 2024-04-22T07:06:08.729 DEBUG:teuthology.orchestra.run.smithi092:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T07:06:08.781 INFO:tasks.workunit.client.0.smithi092.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T07:06:08.783 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T07:06:08.783 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T07:06:09.335 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T07:06:09.393 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T07:06:09.434 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T07:06:09.435 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T07:06:09.435 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T07:06:09.474 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T07:06:09.476 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:06:09.477 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T07:06:09.492 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2024-04-22T07:06:09.494 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2024-04-22T07:06:09.494 DEBUG:teuthology.orchestra.run.smithi092:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86f7587a5a09af35f0895e1a2d08527638fae697 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2024-04-22T07:06:09.555 INFO:tasks.workunit.client.0.smithi092.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2024-04-22T07:06:12.131 INFO:tasks.workunit.client.0.smithi092.stderr:+ mkdir foo 2024-04-22T07:06:12.131 INFO:tasks.workunit.client.0.smithi092.stderr:+ cp linux-2.6.33.tar.bz2 foo 2024-04-22T07:06:12.168 INFO:tasks.workunit.client.0.smithi092.stderr:+ mkdir foo/.snap/barsnap 2024-04-22T07:06:12.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:06:12.170 INFO:tasks.workunit.client.0.smithi092.stderr:mkdir: cannot create directory ‘foo/.snap/barsnap’: No such file or directory 2024-04-22T07:06:12.170 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2024-04-22T07:06:12.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T07:06:12.447 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi092 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86f7587a5a09af35f0895e1a2d08527638fae697 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' 2024-04-22T07:06:12.685 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=af9933ec336b4d789d31a3010eb0362d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi092 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86f7587a5a09af35f0895e1a2d08527638fae697 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' 2024-04-22T07:06:12.687 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-22T07:06:12.697 INFO:tasks.quiescer:joining Quiescers 2024-04-22T07:06:12.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:06:12.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.808+0000 7f52b6752640 1 Processor -- start 2024-04-22T07:06:12.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.808+0000 7f52b6752640 1 -- start start 2024-04-22T07:06:12.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.809+0000 7f52b6752640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 0x7f52b015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.809+0000 7f52b6752640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 0x7f52b00cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.809+0000 7f52b6752640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52b00cbc50 0x7f52b00c7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.809+0000 7f52b6752640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f52b006eb00 2024-04-22T07:06:12.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.809+0000 7f52b6752640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f52b0159250 2024-04-22T07:06:12.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.809+0000 7f52b6752640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b006f0b0 con 0x7f52b00cbc50 2024-04-22T07:06:12.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.809+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 msgr2=0x7f52b00cb710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:12.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.809+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 0x7f52b00cb710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:12.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.809+0000 7f52b4cef640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52b00cbc50 msgr2=0x7f52b00c7630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:12.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.809+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 msgr2=0x7f52b015fab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:12.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.809+0000 7f52b4cef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52b00cbc50 0x7f52b00c7630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:12.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:12.809+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 0x7f52b015fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:13.010+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 msgr2=0x7f52b00cb710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:13.010+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 0x7f52b00cb710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:13.010+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 msgr2=0x7f52b015fab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:13.010+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 0x7f52b015fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:13.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:13.010+0000 7f52b4cef640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52b00cbc50 msgr2=0x7f52b00c7630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:13.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:13.010+0000 7f52b4cef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52b00cbc50 0x7f52b00c7630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:13.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:13.411+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 msgr2=0x7f52b00cb710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:13.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:13.411+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 0x7f52b00cb710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:13.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:13.411+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 msgr2=0x7f52b015fab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:13.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:13.411+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 0x7f52b015fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:13.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:13.411+0000 7f52b4cef640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52b00cbc50 msgr2=0x7f52b00c7630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:13.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:13.411+0000 7f52b4cef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52b00cbc50 0x7f52b00c7630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:14.212+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 msgr2=0x7f52b00cb710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:14.212+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 0x7f52b00cb710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:14.212+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 msgr2=0x7f52b015fab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:14.212+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 0x7f52b015fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:14.212+0000 7f52b4cef640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52b00cbc50 msgr2=0x7f52b00c7630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:14.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:14.212+0000 7f52b4cef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52b00cbc50 0x7f52b00c7630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:15.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:15.810+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52b00cbc50 msgr2=0x7f52b00c7630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:15.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:15.810+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52b00cbc50 0x7f52b00c7630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:15.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:15.810+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 msgr2=0x7f52b015fab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:15.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:15.810+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 0x7f52b015fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:15.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:15.810+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 msgr2=0x7f52b00cb710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:15.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:15.810+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 0x7f52b00cb710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:15.810+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c0073f0 2024-04-22T07:06:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:15.810+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c0036b0 2024-04-22T07:06:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:15.810+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0073f0 legacy=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:15.810+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0036b0 legacy=0x7f529c005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:16.011+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0073f0 legacy=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:16.011+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0036b0 legacy=0x7f529c005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:16.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:16.412+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0073f0 legacy=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:16.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:16.412+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0036b0 legacy=0x7f529c005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:17.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:17.213+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0073f0 legacy=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:17.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:17.213+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0036b0 legacy=0x7f529c005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.810+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0036b0 legacy=0x7f529c005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.810+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0073f0 legacy=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.810+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:06:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.810+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 0x7f529c0098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.810+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 0x7f529c009fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.810+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00d460 con 0x7f529c00c380 2024-04-22T07:06:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.810+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f52b006eb00 2024-04-22T07:06:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.810+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f52b0159250 2024-04-22T07:06:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.810+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.811+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 msgr2=0x7f529c009fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.811+0000 7f52af7fe640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:18.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.811+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 0x7f529c009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:18.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:18.811+0000 7f52af7fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:19.011+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:19.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:19.012+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 msgr2=0x7f529c009fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:19.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:19.012+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 0x7f529c009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:19.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:19.012+0000 7f52af7fe640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:19.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:19.012+0000 7f52af7fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:19.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:19.412+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:19.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:19.413+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 msgr2=0x7f529c009fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:19.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:19.413+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 0x7f529c009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:19.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:19.413+0000 7f52af7fe640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:19.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:19.413+0000 7f52af7fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:20.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:20.213+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:20.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:20.214+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 msgr2=0x7f529c009fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:20.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:20.214+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 0x7f529c009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:20.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:20.214+0000 7f52af7fe640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:20.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:20.214+0000 7f52af7fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:21.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:21.811+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:21.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:21.811+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f52b0159250 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:21.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:21.811+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 msgr2=0x7f529c009fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:21.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:21.811+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52b006eb00 0x7f529c009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:21.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:21.811+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:21.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:21.811+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00a5f0 0x7f529c00aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:21.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:21.811+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c00cf20 2024-04-22T07:06:21.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:21.811+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00a5f0 2024-04-22T07:06:21.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:21.811+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00a5f0 msgr2=0x7f529c00aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:21.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:21.811+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c005b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:21.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:21.811+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00a5f0 0x7f529c00aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:22.012+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c005b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:22.012+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00a5f0 msgr2=0x7f529c00aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:22.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:22.012+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00a5f0 0x7f529c00aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:22.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:22.413+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c005b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:22.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:22.413+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00a5f0 msgr2=0x7f529c00aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:22.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:22.413+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00a5f0 0x7f529c00aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:23.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:23.214+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c005b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:23.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:23.214+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00a5f0 msgr2=0x7f529c00aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:23.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:23.214+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00a5f0 0x7f529c00aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:24.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:24.811+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00a5f0 msgr2=0x7f529c00aa00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:24.811+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00a5f0 0x7f529c00aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:24.811+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c005b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:24.811+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:06:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:24.811+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 0x7f529c013c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:24.811+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00d460 con 0x7f529c00c380 2024-04-22T07:06:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:24.811+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c00cf20 2024-04-22T07:06:24.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:24.811+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c013810 2024-04-22T07:06:24.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:24.811+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:24.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:24.811+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c006260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:24.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:24.811+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 msgr2=0x7f529c013c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:24.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:24.811+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 0x7f529c013c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:25.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:25.012+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:25.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:25.012+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c006260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:25.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:25.012+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 msgr2=0x7f529c013c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:25.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:25.012+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 0x7f529c013c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:25.413+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:25.413+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c006260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:25.413+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 msgr2=0x7f529c013c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:25.413+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 0x7f529c013c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:26.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:26.214+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:26.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:26.214+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c006260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:26.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:26.214+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 msgr2=0x7f529c013c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:26.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:26.214+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 0x7f529c013c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:27.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:27.811+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 msgr2=0x7f529c013c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:27.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:27.811+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 0x7f529c013c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:27.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:27.811+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c006260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:27.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:27.811+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:27.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:27.811+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c009fe0 0x7f529c014250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:27.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:27.811+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006970 0x7f529c0098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:27.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:27.811+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c009fe0 2024-04-22T07:06:27.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:27.811+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c006970 2024-04-22T07:06:27.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:27.811+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c009fe0 msgr2=0x7f529c014250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:27.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:27.811+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c009fe0 0x7f529c014250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:27.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:27.811+0000 7f52af7fe640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006970 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:27.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:27.811+0000 7f52af7fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006970 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:28.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:28.012+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c009fe0 msgr2=0x7f529c014250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:28.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:28.012+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c009fe0 0x7f529c014250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:28.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:28.012+0000 7f52af7fe640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006970 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:28.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:28.012+0000 7f52af7fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006970 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:28.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:28.413+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c009fe0 msgr2=0x7f529c014250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:28.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:28.413+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c009fe0 0x7f529c014250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:28.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:28.413+0000 7f52af7fe640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006970 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:28.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:28.413+0000 7f52af7fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006970 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:29.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:29.214+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c009fe0 msgr2=0x7f529c014250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:29.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:29.214+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c009fe0 0x7f529c014250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:29.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:29.214+0000 7f52af7fe640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006970 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:29.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:29.214+0000 7f52af7fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006970 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:30.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006970 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:30.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006970 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:30.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c009fe0 msgr2=0x7f529c014250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c009fe0 0x7f529c014250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:06:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 0x7f529c013c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c00c380 2024-04-22T07:06:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00d460 con 0x7f529c00cf20 2024-04-22T07:06:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c013810 2024-04-22T07:06:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:30.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:30.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52b4cef640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 msgr2=0x7f529c013c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:30.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:30.812+0000 7f52b4cef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 0x7f529c013c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:31.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:31.013+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:31.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:31.013+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:31.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:31.013+0000 7f52b4cef640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 msgr2=0x7f529c013c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:31.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:31.013+0000 7f52b4cef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 0x7f529c013c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:31.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:31.414+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:31.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:31.414+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:31.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:31.414+0000 7f52b4cef640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 msgr2=0x7f529c013c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:31.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:31.414+0000 7f52b4cef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 0x7f529c013c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:32.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:32.215+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:32.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:32.215+0000 7f52b4cef640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 msgr2=0x7f529c013c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:32.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:32.215+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:32.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:32.215+0000 7f52b4cef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 0x7f529c013c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:33.812+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 msgr2=0x7f529c013c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:33.812+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013810 0x7f529c013c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:33.812+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c003ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:33.812+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:33.812+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c014250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:33.812+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c0051e0 0x7f529c0098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:33.812+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c0051e0 2024-04-22T07:06:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:33.812+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c006d50 2024-04-22T07:06:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:33.812+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c0051e0 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:33.812+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c0051e0 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:33.812+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c014250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:33.812+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c014250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:34.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:34.013+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c0051e0 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:34.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:34.013+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c0051e0 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:34.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:34.013+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c014250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:34.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:34.013+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c014250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:34.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:34.414+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c0051e0 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:34.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:34.414+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c0051e0 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:34.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:34.414+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c014250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:34.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:34.414+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c014250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:35.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:35.215+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c0051e0 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:35.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:35.215+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c0051e0 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:35.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:35.215+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c014250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:35.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:35.215+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c014250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:35.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:06:35.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:06:36.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c014250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:36.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c014250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:36.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c0051e0 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:36.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c0051e0 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:36.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:06:36.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c0098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:36.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 0x7f529c014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:36.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c00cf20 2024-04-22T07:06:36.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00c380 2024-04-22T07:06:36.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c006d50 2024-04-22T07:06:36.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:36.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:36.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:36.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52b4cef640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:36.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:36.812+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:37.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:37.013+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:37.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:37.013+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:37.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:37.013+0000 7f52b4cef640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:37.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:37.013+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:37.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:37.013+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:37.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:37.414+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:37.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:37.414+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:37.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:37.414+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:37.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:37.414+0000 7f52b4cef640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:37.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:37.414+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:38.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:38.215+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:38.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:38.215+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:38.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:38.215+0000 7f52b4cef640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:38.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:38.215+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:38.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:38.215+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:39.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:39.813+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:39.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:39.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:39.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:39.813+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:39.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:39.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:39.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:39.813+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00cf20 legacy=0x7f529c00c7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:39.813+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:39.813+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c005b30 2024-04-22T07:06:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:39.813+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c013c10 2024-04-22T07:06:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:39.813+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005b30 legacy=0x7f529c005f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:39.813+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:39.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:39.813+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:40.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:40.014+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005b30 legacy=0x7f529c005f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:40.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:40.014+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:40.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:40.014+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:40.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:40.415+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005b30 legacy=0x7f529c005f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:40.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:40.415+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:40.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:40.415+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:41.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:41.216+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005b30 legacy=0x7f529c005f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:41.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:41.216+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:41.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:41.216+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:42.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:42.813+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:42.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:42.813+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:42.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:42.813+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005b30 legacy=0x7f529c005f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:42.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:42.813+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:06:42.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:42.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:42.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:42.813+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c006d50 2024-04-22T07:06:42.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:42.813+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c00c380 2024-04-22T07:06:42.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:42.813+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00cf20 2024-04-22T07:06:42.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:42.813+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:42.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:42.813+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:42.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:42.813+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:42.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:42.813+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:43.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:43.014+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:43.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:43.014+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:43.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:43.014+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:43.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:43.014+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:43.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:43.415+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:43.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:43.415+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:43.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:43.415+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:43.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:43.415+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:44.216+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:44.216+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:44.216+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:44.216+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:45.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:45.813+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:45.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:45.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:45.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:45.813+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:45.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:45.813+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:45.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:45.813+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:45.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:45.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c004ad0 0x7f529c005d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:45.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:45.813+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c004ad0 2024-04-22T07:06:45.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:45.813+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c013c10 2024-04-22T07:06:45.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:45.813+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c004ad0 msgr2=0x7f529c005d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:45.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:45.813+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c004ad0 0x7f529c005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:45.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:45.813+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:45.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:45.813+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:46.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:46.014+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c004ad0 msgr2=0x7f529c005d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:46.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:46.014+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c004ad0 0x7f529c005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:46.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:46.014+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:46.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:46.014+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:46.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:46.415+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c004ad0 msgr2=0x7f529c005d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:46.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:46.415+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c004ad0 0x7f529c005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:46.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:46.415+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:46.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:46.415+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:47.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:47.215+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c004ad0 msgr2=0x7f529c005d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:47.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:47.215+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c004ad0 0x7f529c005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:47.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:47.215+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:47.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:47.215+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:48.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:48.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:48.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c004ad0 msgr2=0x7f529c005d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:48.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c004ad0 0x7f529c005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:48.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:06:48.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:48.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:48.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00d460 con 0x7f529c006d50 2024-04-22T07:06:48.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00cf20 2024-04-22T07:06:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c013c10 2024-04-22T07:06:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52b4cef640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:48.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:48.813+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:49.014+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:49.014+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:49.014+0000 7f52b4cef640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:49.014+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:49.014+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:49.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:49.415+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:49.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:49.415+0000 7f52b4cef640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:49.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:49.415+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:49.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:49.415+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:49.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:49.415+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:50.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:50.216+0000 7f52b4cef640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:50.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:50.216+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:50.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:50.216+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:50.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:50.216+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:50.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:50.216+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:51.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:51.813+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:51.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:51.813+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013c10 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:51.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:51.813+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:51.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:51.813+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:51.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:51.813+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c005b30 2024-04-22T07:06:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:51.813+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00c380 2024-04-22T07:06:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:51.813+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:51.813+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005b30 legacy=0x7f529c005f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:51.813+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:52.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:52.014+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005b30 legacy=0x7f529c005f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:52.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:52.014+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:52.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:52.014+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:52.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:52.415+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005b30 legacy=0x7f529c005f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:52.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:52.415+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:52.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:52.415+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:53.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:53.216+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:53.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:53.216+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005b30 legacy=0x7f529c005f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:53.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:53.216+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:54.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:54.813+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:54.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:54.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00c380 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:54.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:54.813+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005b30 legacy=0x7f529c005f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:54.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:54.813+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:06:54.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:54.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:54.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:54.813+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00d460 con 0x7f529c004ad0 2024-04-22T07:06:54.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:54.813+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c006d50 2024-04-22T07:06:54.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:54.813+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00cf20 2024-04-22T07:06:54.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:54.813+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c018840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:54.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:54.813+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:54.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:54.813+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:54.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:54.813+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:55.014+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c018840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:55.014+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:55.014+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:55.014+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:55.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:55.415+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c018840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:55.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:55.415+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:55.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:55.415+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:55.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:55.415+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:56.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:56.216+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c018840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:06:56.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:56.216+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:06:56.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:56.216+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:06:56.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:56.216+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:57.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:57.813+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:57.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:57.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:57.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:57.813+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c006d50 legacy=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:57.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:57.813+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c018840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:57.813+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:57.813+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00c380 0x7f529c0098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:57.813+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c013c10 2024-04-22T07:06:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:57.813+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c00c380 2024-04-22T07:06:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:57.813+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:57.814+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:57.814+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00c380 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:57.814+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00c380 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:58.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:58.015+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:58.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:58.015+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:58.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:58.015+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00c380 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:58.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:58.015+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00c380 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:58.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:58.416+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:58.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:58.416+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:58.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:58.416+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00c380 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:58.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:58.416+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00c380 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:59.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:59.217+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:06:59.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:59.217+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:59.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:59.217+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00c380 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:06:59.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:06:59.217+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00c380 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00c380 msgr2=0x7f529c0098b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00c380 0x7f529c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:00.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:07:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c004ad0 2024-04-22T07:07:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c013c10 2024-04-22T07:07:00.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c00cf20 2024-04-22T07:07:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:00.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:00.814+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:01.015+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:01.015+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:01.015+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:01.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:01.015+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:01.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:01.015+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:01.416+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:01.416+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:01.416+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:01.416+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:01.416+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:02.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:02.217+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:02.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:02.217+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:02.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:02.217+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:02.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:02.217+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:02.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:02.217+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:03.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:03.814+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:03.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:03.814+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:03.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:03.814+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:03.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:03.814+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:03.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:03.814+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:03.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:03.814+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:03.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:03.814+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00d460 con 0x7f529c00c380 2024-04-22T07:07:03.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:03.814+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c006d50 2024-04-22T07:07:03.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:03.814+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:03.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:03.814+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:03.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:03.814+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:04.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:04.015+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:04.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:04.015+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:04.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:04.015+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:04.416+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:04.416+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:04.416+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:05.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:05.217+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:05.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:05.217+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:05.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:05.217+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:05.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:07:05.490 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:07:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:07:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:06.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c004ad0 2024-04-22T07:07:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c013c10 2024-04-22T07:07:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c00cf20 2024-04-22T07:07:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:06.814+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:07.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:07.015+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:07.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:07.015+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:07.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:07.015+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:07.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:07.015+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:07.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:07.015+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:07.416+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:07.416+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:07.416+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:07.416+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:07.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:07.416+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:08.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:08.217+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:08.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:08.217+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:08.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:08.217+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:08.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:08.217+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:08.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:08.217+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:09.815+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:09.815+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00cf20 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:09.815+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:09.815+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:09.815+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:09.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:09.815+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:09.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:09.815+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00d460 con 0x7f529c00c380 2024-04-22T07:07:09.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:09.815+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c006d50 2024-04-22T07:07:09.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:09.815+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:09.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:09.815+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:09.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:09.815+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:10.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:10.016+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:10.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:10.016+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:10.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:10.016+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:10.417+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:10.417+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:10.417+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:11.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:11.218+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:11.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:11.218+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:11.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:11.218+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:12.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:12.815+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:12.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:12.815+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:12.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:12.815+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00c380 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:12.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:12.815+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:07:12.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:12.815+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:12.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:12.815+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c004ad0 2024-04-22T07:07:12.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:12.815+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00d460 con 0x7f529c0098b0 2024-04-22T07:07:12.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:12.815+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c006d50 2024-04-22T07:07:12.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:12.815+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:12.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:12.815+0000 7f52b4cef640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:12.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:12.815+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c00b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:12.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:12.815+0000 7f52b4cef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:13.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:13.016+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:13.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:13.016+0000 7f52b4cef640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:13.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:13.016+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c00b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:13.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:13.016+0000 7f52b4cef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:13.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:13.417+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:13.417+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c00b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:13.417+0000 7f52b4cef640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:13.417+0000 7f52b4cef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:14.218+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:14.218+0000 7f52b4cef640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:14.218+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c00b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:14.218+0000 7f52b4cef640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:15.815+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:15.815+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:15.815+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c00b820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:15.815+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c004ad0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:15.815+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:15.815+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:15.815+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00cef0 2024-04-22T07:07:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:15.815+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c013c10 2024-04-22T07:07:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:15.815+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:15.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:15.815+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:15.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:15.815+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:15.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:15.815+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:16.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:16.016+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:16.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:16.016+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:16.016+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:16.016+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:16.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:16.417+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:16.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:16.417+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:16.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:16.417+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:16.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:16.417+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:17.218+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:17.218+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:17.218+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:17.218+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:07:18.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:18.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c0098b0 2024-04-22T07:07:18.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00cef0 2024-04-22T07:07:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c013c10 2024-04-22T07:07:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:18.816+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:19.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:19.017+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:19.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:19.017+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:19.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:19.017+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:19.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:19.017+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:19.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:19.017+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:19.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:19.418+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:19.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:19.418+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:19.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:19.418+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:19.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:19.418+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:19.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:19.418+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:20.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:20.219+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:20.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:20.219+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:20.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:20.219+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:20.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:20.219+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:20.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:20.219+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:21.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:21.816+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:21.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:21.816+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:21.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:21.816+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:21.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:21.816+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00cef0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:21.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:21.816+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:21.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:21.816+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:21.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:21.816+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c005f10 2024-04-22T07:07:21.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:21.816+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c006d50 2024-04-22T07:07:21.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:21.816+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005f10 legacy=0x7f529c00b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:21.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:21.816+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:21.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:21.816+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:22.017+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005f10 legacy=0x7f529c00b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:22.017+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:22.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:22.017+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:22.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:22.418+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005f10 legacy=0x7f529c00b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:22.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:22.418+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:22.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:22.418+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:23.218+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005f10 legacy=0x7f529c00b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:23.218+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:23.218+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c005f10 legacy=0x7f529c00b820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:07:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c0098b0 2024-04-22T07:07:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c006d50 2024-04-22T07:07:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c013c10 2024-04-22T07:07:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:24.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:24.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:24.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:24.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:24.815+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:25.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:25.016+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:25.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:25.016+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:25.016+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:25.016+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:25.016+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:25.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:25.417+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:25.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:25.417+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:25.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:25.417+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:25.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:25.417+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:25.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:25.417+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:26.218+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:26.218+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:26.218+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:26.218+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:26.218+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:27.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:27.816+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:27.816+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:27.816+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:27.816+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:27.816+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0098b0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:27.816+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:27.816+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c003ac0 2024-04-22T07:07:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:27.816+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00ced0 2024-04-22T07:07:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:27.816+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:27.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:27.816+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c003ac0 legacy=0x7f529c003ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:27.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:27.816+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:28.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:28.017+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:28.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:28.017+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c003ac0 legacy=0x7f529c003ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:28.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:28.017+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:28.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:28.418+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:28.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:28.418+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c003ac0 legacy=0x7f529c003ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:28.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:28.418+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:29.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:29.219+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c003ac0 legacy=0x7f529c003ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:29.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:29.219+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:29.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:29.219+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:30.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:30.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:30.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c003ac0 legacy=0x7f529c003ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:30.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:07:30.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:30.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:30.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c003ac0 2024-04-22T07:07:30.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c006d50 2024-04-22T07:07:30.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c00ced0 2024-04-22T07:07:30.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:30.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:30.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c003ac0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:30.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:30.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:30.816+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:31.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:31.017+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c003ac0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:31.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:31.017+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:31.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:31.017+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:31.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:31.017+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:31.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:31.017+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:31.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:31.418+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:31.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:31.418+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c003ac0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:31.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:31.418+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:31.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:31.418+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:31.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:31.418+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:32.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:32.219+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c003ac0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:32.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:32.219+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:32.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:32.219+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:32.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:32.219+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:32.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:32.219+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:33.816+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:33.816+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:33.816+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:33.816+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c006d50 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:33.816+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c003ac0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:33.816+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:33.816+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c009890 2024-04-22T07:07:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:33.816+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c013c10 2024-04-22T07:07:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:33.816+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:33.816+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:33.816+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c009890 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:34.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:34.017+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:34.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:34.017+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c009890 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:34.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:34.017+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:34.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:34.418+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c009890 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:34.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:34.418+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:34.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:34.418+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:35.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:35.219+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:35.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:35.219+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c009890 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:35.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:35.219+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:35.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:07:35.759 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:07:36.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:36.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c013c10 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:36.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c009890 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:36.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:07:36.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:36.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:36.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c0045b0 2024-04-22T07:07:36.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c006d50 2024-04-22T07:07:36.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c003ac0 2024-04-22T07:07:36.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0045b0 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:36.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:36.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:36.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:36.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:36.817+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:37.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:37.018+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0045b0 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:37.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:37.018+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:37.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:37.018+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:37.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:37.018+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:37.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:37.018+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:37.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:37.419+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:37.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:37.419+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0045b0 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:37.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:37.419+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:37.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:37.419+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:37.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:37.419+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:38.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:38.220+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:38.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:38.220+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0045b0 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:38.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:38.220+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:38.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:38.220+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:38.220+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:39.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:39.817+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:39.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:39.817+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:39.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:39.817+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:39.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:39.817+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:39.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:39.817+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0045b0 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:39.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:39.817+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00ced0 0x7f529c014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:39.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:39.817+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c013c10 2024-04-22T07:07:39.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:39.817+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00ced0 2024-04-22T07:07:39.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:39.817+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:39.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:39.817+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013c10 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:39.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:39.817+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00ced0 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:40.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:40.018+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:40.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:40.018+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013c10 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:40.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:40.018+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00ced0 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:40.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:40.419+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013c10 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:40.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:40.419+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:40.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:40.419+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00ced0 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:41.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:41.220+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:41.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:41.220+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013c10 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:41.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:41.220+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00ced0 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00ced0 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013c10 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:07:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:42.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c0045b0 2024-04-22T07:07:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c006d50 2024-04-22T07:07:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c003ac0 2024-04-22T07:07:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0045b0 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:42.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:42.817+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:43.018+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:43.018+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:43.018+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0045b0 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:43.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:43.018+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:43.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:43.018+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:43.419+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:43.419+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0045b0 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:43.419+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:43.419+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:43.419+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:44.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:44.220+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:44.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:44.220+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:44.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:44.220+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0045b0 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:44.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:44.220+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:44.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:44.220+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:45.818+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:45.818+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:45.818+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:45.818+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c006d50 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:45.818+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c0045b0 legacy=0x7f529c003ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:45.818+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:45.818+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c013bf0 2024-04-22T07:07:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:45.818+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00ced0 2024-04-22T07:07:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:45.818+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:45.818+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:45.818+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:46.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:46.019+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:46.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:46.019+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:46.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:46.019+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:46.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:46.420+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:46.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:46.420+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:46.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:46.420+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:47.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:47.221+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:47.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:47.221+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:47.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:47.221+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:48.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:48.818+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 msgr2=0x7f529c014230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:48.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:48.818+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c00ced0 0x7f529c014230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:48.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:48.818+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:48.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:48.818+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:48.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:48.818+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:07:48.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:48.818+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c006d50 2024-04-22T07:07:48.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:48.818+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c00ced0 2024-04-22T07:07:48.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:48.818+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c003ac0 2024-04-22T07:07:48.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:48.818+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:48.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:48.818+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:48.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:48.818+0000 7f52b4cef640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00ced0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:48.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:48.818+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:49.019+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:49.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:49.019+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:49.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:49.019+0000 7f52b4cef640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00ced0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:49.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:49.019+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:49.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:49.420+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:49.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:49.420+0000 7f52b4cef640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00ced0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:49.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:49.420+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:49.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:49.420+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:50.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:50.221+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:50.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:50.221+0000 7f52b4cef640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00ced0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:50.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:50.221+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:50.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:50.221+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:51.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:51.818+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:51.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:51.818+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:51.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:51.818+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c00ced0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:51.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:51.818+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:51.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:51.818+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013bf0 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:51.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:51.818+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00b820 0x7f529c012740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:51.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:51.818+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00b820 2024-04-22T07:07:51.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:51.818+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c013bf0 2024-04-22T07:07:51.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:51.818+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c012740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:51.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:51.818+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00b820 0x7f529c012740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:51.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:51.818+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013bf0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:51.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:51.818+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013bf0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:52.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:52.019+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c012740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:52.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:52.019+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00b820 0x7f529c012740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:52.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:52.019+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013bf0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:52.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:52.019+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013bf0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:52.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:52.420+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c012740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:52.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:52.420+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00b820 0x7f529c012740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:52.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:52.420+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013bf0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:52.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:52.420+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013bf0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:53.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:53.221+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c012740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:53.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:53.221+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00b820 0x7f529c012740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:53.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:53.221+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013bf0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:53.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:53.221+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013bf0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:54.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013bf0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:54.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c013bf0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:54.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c012740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:54.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00b820 0x7f529c012740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:54.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:07:54.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:54.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:54.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00d460 con 0x7f529c006d50 2024-04-22T07:07:54.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00ced0 2024-04-22T07:07:54.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c003ac0 2024-04-22T07:07:54.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:54.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:54.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:54.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:54.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:54.819+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:55.020+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:55.020+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:55.020+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:55.020+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:55.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:55.020+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:55.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:55.421+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:55.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:55.421+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:55.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:55.421+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:55.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:55.421+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:55.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:55.421+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:56.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:56.222+0000 7f52b4cef640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:07:56.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:56.222+0000 7f52b4cef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:56.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:56.222+0000 7f52affff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:07:56.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:56.222+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:07:56.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:56.222+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:57.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:57.819+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:57.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:57.819+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c003ac0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:57.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:57.819+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:57.819+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:57.819+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:57.819+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c017250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:57.819+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c013bf0 2024-04-22T07:07:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:57.819+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00b820 2024-04-22T07:07:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:57.819+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:57.819+0000 7f52af7fe640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c017250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:57.819+0000 7f52af7fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c017250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:58.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:58.019+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:58.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:58.019+0000 7f52af7fe640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c017250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:58.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:58.019+0000 7f52af7fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c017250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:58.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:58.420+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:58.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:58.420+0000 7f52af7fe640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c017250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:58.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:58.420+0000 7f52af7fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c017250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:59.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:59.221+0000 7f52af7fe640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c017250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:07:59.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:59.221+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:07:59.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:07:59.221+0000 7f52af7fe640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c017250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:00.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:00.818+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c017250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:00.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:00.818+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c017250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:00.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:00.818+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:00.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:00.819+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:08:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:00.819+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c00c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:00.819+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00d460 con 0x7f529c006d50 2024-04-22T07:08:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:00.819+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006f390 con 0x7f529c013bf0 2024-04-22T07:08:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:00.819+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00b820 2024-04-22T07:08:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:00.819+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:00.819+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:00.819+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:00.819+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:01.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:01.020+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:01.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:01.020+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:01.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:01.020+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:01.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:01.020+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:01.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:01.421+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:01.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:01.421+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:01.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:01.421+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:01.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:01.421+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:02.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:02.222+0000 7f52af7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:02.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:02.222+0000 7f52affff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:02.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:02.222+0000 7f52b4cef640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:02.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:02.222+0000 7f52b4cef640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:03.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:03.819+0000 7f52adffb640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:03.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:03.819+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:03.819+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:03.819+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:03.819+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:03.819+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 0x7f529c00bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:03.819+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00ced0 2024-04-22T07:08:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:03.819+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c004ad0 2024-04-22T07:08:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:03.819+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:03.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:03.819+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:03.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:03.819+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 msgr2=0x7f529c00bc00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:03.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:03.819+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 0x7f529c00bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:04.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:04.020+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:04.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:04.020+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:04.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:04.020+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 msgr2=0x7f529c00bc00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:04.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:04.020+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 0x7f529c00bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:04.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:04.421+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:04.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:04.421+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:04.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:04.421+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 msgr2=0x7f529c00bc00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:04.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:04.421+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 0x7f529c00bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:05.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:05.222+0000 7f52affff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:05.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:05.222+0000 7f52affff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:05.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:05.222+0000 7f52af7fe640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 msgr2=0x7f529c00bc00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:05.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:05.222+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 0x7f529c00bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:05.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:08:05.896 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:08:06.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 msgr2=0x7f529c00bc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:06.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 0x7f529c00bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:06.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52adffb640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:06.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:06.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 0x7f529c00bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:06.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52b4cef640 1 -- reap_dead start 2024-04-22T07:08:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52adffb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00c160 con 0x7f529c006d50 2024-04-22T07:08:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52adffb640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c00d460 con 0x7f529c013bf0 2024-04-22T07:08:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52adffb640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c004ad0 2024-04-22T07:08:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 msgr2=0x7f529c00bc00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:06.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:06.819+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 0x7f529c00bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:07.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:07.020+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:07.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:07.020+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:07.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:07.020+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 msgr2=0x7f529c00bc00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:07.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:07.020+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 0x7f529c00bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:07.421+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:07.421+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 msgr2=0x7f529c00bc00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:07.421+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:07.421+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 0x7f529c00bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:08.222+0000 7f52b4cef640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:08.222+0000 7f52af7fe640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:08.222+0000 7f52affff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 msgr2=0x7f529c00bc00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:08.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:08.222+0000 7f52affff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 0x7f529c00bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:09.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:09.820+0000 7f52adffb640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 msgr2=0x7f529c00bc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:09.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:09.820+0000 7f52adffb640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529c004ad0 0x7f529c00bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:09.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:09.820+0000 7f52adffb640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7f529c013bf0 legacy=0x7f529c00b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:09.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:09.820+0000 7f52adffb640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f529c006d50 legacy=0x7f529c005f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:09.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:09.820+0000 7f52adffb640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c0055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:09.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:09.820+0000 7f52adffb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c00c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:09.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:09.820+0000 7f52adffb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7f52b015fff0 con 0x7f529c00ced0 2024-04-22T07:08:09.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:09.820+0000 7f52adffb640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f52b00c6ec0 con 0x7f529c00b820 2024-04-22T07:08:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:09.820+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:09.820+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:09.820+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:09.820+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:10.021+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:10.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:10.021+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:10.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:10.021+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:10.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:10.021+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:10.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:10.422+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:10.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:10.422+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:10.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:10.422+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:10.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:10.422+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:11.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:11.223+0000 7f52af7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 msgr2=0x7f529c00c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:11.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:11.223+0000 7f52af7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f529c00ced0 0x7f529c00c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:11.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:11.223+0000 7f52affff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 msgr2=0x7f529c0055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:11.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:11.223+0000 7f52affff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f529c00b820 0x7f529c0055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:12.739 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-22T07:08:12.740 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2024-04-22T07:08:12.754 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-22T07:08:12.764 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-22T07:08:12.764 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-22T07:08:12.764 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T07:08:12.798 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056106,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19034 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-04-22T07:08:12.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:08:12.798 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-04-22T07:08:12.855 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-22T07:08:12.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:08:12.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:08:12.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:08:12.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2024-04-22T07:08:13.023 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-22T07:08:13.034 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-22T07:08:13.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:08:13.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.137+0000 7fdfd89a4640 1 Processor -- start 2024-04-22T07:08:13.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.137+0000 7fdfd89a4640 1 -- start start 2024-04-22T07:08:13.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.138+0000 7fdfd89a4640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4151140 0x7fdfd4156f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:13.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.138+0000 7fdfd89a4640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfd40c6ec0 0x7fdfd40c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:13.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.138+0000 7fdfd89a4640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfd406f390 2024-04-22T07:08:13.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.138+0000 7fdfd89a4640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfd40c6ec0 2024-04-22T07:08:13.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.138+0000 7fdfd89a4640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfd4151140 2024-04-22T07:08:13.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.138+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4151140 msgr2=0x7fdfd4156f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:13.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.138+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfd40c6ec0 msgr2=0x7fdfd40c7320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:13.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.138+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfd406f390 legacy=0x7fdfd406f820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:13.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.138+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4151140 0x7fdfd4156f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:13.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.138+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfd40c6ec0 0x7fdfd40c7320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:13.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.339+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfd406f390 legacy=0x7fdfd406f820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:13.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.339+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfd40c6ec0 msgr2=0x7fdfd40c7320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:13.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.339+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4151140 msgr2=0x7fdfd4156f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:13.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.339+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfd40c6ec0 0x7fdfd40c7320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.339+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4151140 0x7fdfd4156f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:13.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.740+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4151140 msgr2=0x7fdfd4156f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:13.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.740+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfd406f390 legacy=0x7fdfd406f820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:13.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.740+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfd40c6ec0 msgr2=0x7fdfd40c7320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:13.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.740+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4151140 0x7fdfd4156f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:13.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:13.740+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfd40c6ec0 0x7fdfd40c7320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:14.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:14.541+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfd406f390 legacy=0x7fdfd406f820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:14.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:14.541+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4151140 msgr2=0x7fdfd4156f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:14.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:14.541+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfd40c6ec0 msgr2=0x7fdfd40c7320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:14.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:14.541+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4151140 0x7fdfd4156f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:14.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:14.541+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfd40c6ec0 0x7fdfd40c7320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:16.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.139+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4151140 msgr2=0x7fdfd4156f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:16.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.139+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4151140 0x7fdfd4156f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:16.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.139+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfd40c6ec0 msgr2=0x7fdfd40c7320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:16.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.139+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfd40c6ec0 0x7fdfd40c7320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:16.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.139+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfd406f390 legacy=0x7fdfd406f820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:16.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.139+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4009610 0x7fdfb400bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:16.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.139+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca8d0 con 0x7fdfb40058d0 2024-04-22T07:08:16.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.139+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb4009610 2024-04-22T07:08:16.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.139+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40058d0 legacy=0x7fdfb4007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:16.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.139+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4009610 msgr2=0x7fdfb400bad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:16.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.139+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4009610 0x7fdfb400bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:16.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.340+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4009610 msgr2=0x7fdfb400bad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:16.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.340+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40058d0 legacy=0x7fdfb4007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:16.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.340+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4009610 0x7fdfb400bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:16.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.741+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4009610 msgr2=0x7fdfb400bad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:16.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.741+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40058d0 legacy=0x7fdfb4007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:16.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:16.741+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4009610 0x7fdfb400bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:17.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:17.542+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40058d0 legacy=0x7fdfb4007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:17.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:17.542+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4009610 msgr2=0x7fdfb400bad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:17.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:17.542+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4009610 0x7fdfb400bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:19.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.139+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4009610 msgr2=0x7fdfb400bad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:19.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.139+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4009610 0x7fdfb400bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:19.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.139+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40058d0 legacy=0x7fdfb4007d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.139+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:08:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.139+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 0x7fdfb400ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.139+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb400cd20 2024-04-22T07:08:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.139+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca8d0 con 0x7fdfd406f390 2024-04-22T07:08:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.139+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400c900 2024-04-22T07:08:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.139+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400cd20 legacy=0x7fdfb4007cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.139+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 msgr2=0x7fdfb400ba50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.139+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfd406f390 legacy=0x7fdfb400c1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.139+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 0x7fdfb400ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:19.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.340+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400cd20 legacy=0x7fdfb4007cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:19.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.340+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfd406f390 legacy=0x7fdfb400c1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:19.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.340+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 msgr2=0x7fdfb400ba50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:19.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.340+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 0x7fdfb400ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.741+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400cd20 legacy=0x7fdfb4007cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.741+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 msgr2=0x7fdfb400ba50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.741+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfd406f390 legacy=0x7fdfb400c1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:19.741+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 0x7fdfb400ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:20.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:20.542+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400cd20 legacy=0x7fdfb4007cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:20.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:20.542+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 msgr2=0x7fdfb400ba50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:20.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:20.542+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfd406f390 legacy=0x7fdfb400c1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:20.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:20.542+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 0x7fdfb400ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:22.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.139+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 msgr2=0x7fdfb400ba50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:22.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.139+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 0x7fdfb400ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:22.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.139+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfd406f390 legacy=0x7fdfb400c1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:22.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.139+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400cd20 legacy=0x7fdfb4007cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:22.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.139+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40089c0 0x7fdfb4008da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:22.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.139+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40133b0 0x7fdfb40157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:22.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.139+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40133b0 2024-04-22T07:08:22.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.139+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb40089c0 2024-04-22T07:08:22.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.139+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40133b0 msgr2=0x7fdfb40157a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:22.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.139+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40133b0 0x7fdfb40157a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:22.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.139+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40089c0 msgr2=0x7fdfb4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:22.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.139+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40089c0 0x7fdfb4008da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:22.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.340+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40133b0 msgr2=0x7fdfb40157a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:22.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.340+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40133b0 0x7fdfb40157a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:22.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.340+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40089c0 msgr2=0x7fdfb4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:22.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.340+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40089c0 0x7fdfb4008da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.741+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40133b0 msgr2=0x7fdfb40157a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:22.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.741+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40133b0 0x7fdfb40157a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.741+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40089c0 msgr2=0x7fdfb4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:22.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:22.741+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40089c0 0x7fdfb4008da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:23.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:23.542+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40133b0 msgr2=0x7fdfb40157a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:23.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:23.542+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40133b0 0x7fdfb40157a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:23.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:23.542+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40089c0 msgr2=0x7fdfb4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:23.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:23.542+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40089c0 0x7fdfb4008da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:25.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40089c0 msgr2=0x7fdfb4008da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:25.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40089c0 0x7fdfb4008da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:25.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40133b0 msgr2=0x7fdfb40157a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:25.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40133b0 0x7fdfb40157a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:25.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:08:25.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 0x7fdfb400ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:25.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 0x7fdfb4006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:25.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb40086b0 con 0x7fdfb400c1f0 2024-04-22T07:08:25.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40157a0 2024-04-22T07:08:25.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb400c900 2024-04-22T07:08:25.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:25.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:25.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb400ba50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:25.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c1f0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:25.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.140+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 0x7fdfb400ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.341+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.341+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.341+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb400ba50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.341+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c1f0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.341+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 0x7fdfb400ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:25.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.742+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:25.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.742+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:25.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.742+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c1f0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:25.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.742+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb400ba50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:25.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:25.742+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 0x7fdfb400ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:26.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:26.543+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:26.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:26.543+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:26.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:26.543+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb400ba50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:26.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:26.543+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c1f0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:26.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:26.543+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 0x7fdfb400ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:28.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.140+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb400ba50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:28.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.140+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 0x7fdfb400ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:28.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.140+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:28.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.140+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:28.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.140+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c1f0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:28.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.140+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4007200 0x7fdfb40075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:28.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.140+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb4007b20 2024-04-22T07:08:28.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.140+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb4007200 2024-04-22T07:08:28.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.140+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4007b20 legacy=0x7fdfb4015d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:28.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.140+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4007200 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:28.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.140+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4007200 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:28.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.341+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4007b20 legacy=0x7fdfb4015d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:28.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.341+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4007200 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:28.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.341+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4007200 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:28.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.742+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4007b20 legacy=0x7fdfb4015d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:28.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.742+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4007200 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:28.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:28.742+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4007200 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:29.543+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4007b20 legacy=0x7fdfb4015d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:29.543+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4007200 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:29.543+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4007200 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:31.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4007200 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:31.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4007200 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:31.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4007b20 legacy=0x7fdfb4015d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:31.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:08:31.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 0x7fdfb4006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:31.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 0x7fdfb40075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:31.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb40086b0 con 0x7fdfb400c1f0 2024-04-22T07:08:31.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40157a0 2024-04-22T07:08:31.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb400c900 2024-04-22T07:08:31.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c1f0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:31.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:31.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:31.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:31.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.140+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:31.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.341+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:31.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.341+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c1f0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:31.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.341+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:31.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.341+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:31.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.341+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:31.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.742+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:31.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.742+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:31.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.742+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c1f0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:31.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.742+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:31.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:31.742+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:32.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:32.543+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:32.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:32.543+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c1f0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:32.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:32.543+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:32.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:32.543+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:32.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:32.543+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:34.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.140+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:34.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.140+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c900 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:34.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.140+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:34.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.140+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40157a0 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:34.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.140+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c1f0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:34.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.140+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012ab0 0x7fdfb4017c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:34.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.140+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb400ba30 2024-04-22T07:08:34.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.140+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb4012ab0 2024-04-22T07:08:34.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.140+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012ab0 msgr2=0x7fdfb4017c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:34.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.140+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400ba30 legacy=0x7fdfb4011390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:34.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.140+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012ab0 0x7fdfb4017c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:34.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.341+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012ab0 msgr2=0x7fdfb4017c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:34.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.341+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400ba30 legacy=0x7fdfb4011390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:34.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.341+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012ab0 0x7fdfb4017c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:34.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.742+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012ab0 msgr2=0x7fdfb4017c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:34.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.742+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400ba30 legacy=0x7fdfb4011390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:34.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:34.742+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012ab0 0x7fdfb4017c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:35.543+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012ab0 msgr2=0x7fdfb4017c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:35.543+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012ab0 0x7fdfb4017c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:35.543+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400ba30 legacy=0x7fdfb4011390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:36.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:08:36.024 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:08:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.140+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012ab0 msgr2=0x7fdfb4017c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.140+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012ab0 0x7fdfb4017c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.140+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400ba30 legacy=0x7fdfb4011390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.140+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:08:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.140+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 0x7fdfb4006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.140+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 0x7fdfb40075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.140+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb40086b0 con 0x7fdfb400c900 2024-04-22T07:08:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.140+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400ba00 2024-04-22T07:08:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.140+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb40157a0 2024-04-22T07:08:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.140+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c900 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.141+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.141+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.141+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.141+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:37.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.342+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c900 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:37.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.342+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:37.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.342+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:37.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.342+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:37.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.342+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:37.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.743+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:37.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.743+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c900 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:37.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.743+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:37.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.743+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:37.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:37.743+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:38.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:38.544+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:38.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:38.544+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c900 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:38.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:38.544+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:38.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:38.544+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:38.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:38.544+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:40.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.141+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:40.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.141+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:40.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.141+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 msgr2=0x7fdfb40075e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:40.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.141+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 0x7fdfb40075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.141+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c900 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.141+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb400f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.141+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb400f8c0 2024-04-22T07:08:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.141+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400c150 2024-04-22T07:08:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.141+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb400f380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.141+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400f8c0 legacy=0x7fdfb4011cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:40.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.141+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb400f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:40.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.342+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400f8c0 legacy=0x7fdfb4011cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.342+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb400f380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.342+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb400f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:40.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.743+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400f8c0 legacy=0x7fdfb4011cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:40.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.743+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb400f380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:40.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:40.743+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb400f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:41.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:41.544+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400f8c0 legacy=0x7fdfb4011cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:41.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:41.544+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb400f380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:41.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:41.544+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb400f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:43.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb400f380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:43.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb400f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:43.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400f8c0 legacy=0x7fdfb4011cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:43.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:08:43.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c900 0x7fdfb4006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:43.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 0x7fdfb400f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:43.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb40086b0 con 0x7fdfb400c150 2024-04-22T07:08:43.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400c900 2024-04-22T07:08:43.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb40157a0 2024-04-22T07:08:43.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:43.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:43.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c900 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:43.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfd2d9d640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 msgr2=0x7fdfb400f380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:43.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.141+0000 7fdfd2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 0x7fdfb400f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.342+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.342+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.342+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c900 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.342+0000 7fdfd2d9d640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 msgr2=0x7fdfb400f380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:43.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.342+0000 7fdfd2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 0x7fdfb400f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.743+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.743+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.743+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c900 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.743+0000 7fdfd2d9d640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 msgr2=0x7fdfb400f380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:43.743+0000 7fdfd2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 0x7fdfb400f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:44.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:44.544+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:44.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:44.544+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c900 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:44.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:44.544+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:44.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:44.544+0000 7fdfd2d9d640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 msgr2=0x7fdfb400f380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:44.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:44.544+0000 7fdfd2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 0x7fdfb400f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:46.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.141+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 msgr2=0x7fdfb400f380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:46.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.141+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40157a0 0x7fdfb400f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:46.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.141+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c900 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:46.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.141+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c900 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:46.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.141+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:46.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.141+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:46.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.141+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb400fca0 2024-04-22T07:08:46.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.141+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40075e0 2024-04-22T07:08:46.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.142+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400fca0 legacy=0x7fdfb4012160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:46.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.142+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:46.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.142+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:46.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.343+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:46.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.343+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400fca0 legacy=0x7fdfb4012160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:46.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.343+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:46.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.744+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400fca0 legacy=0x7fdfb4012160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:46.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.744+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:46.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:46.744+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:47.545+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400fca0 legacy=0x7fdfb4012160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:47.545+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:47.545+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:49.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.142+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:49.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.142+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:49.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.142+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400fca0 legacy=0x7fdfb4012160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:49.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.142+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:08:49.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.142+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 0x7fdfb400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:49.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.142+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb40086b0 con 0x7fdfb400c150 2024-04-22T07:08:49.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.142+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb40075e0 2024-04-22T07:08:49.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.142+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400c900 2024-04-22T07:08:49.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.142+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:49.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.142+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:49.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.142+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:49.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.142+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:49.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.343+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:49.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.343+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:49.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.343+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:49.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.343+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:49.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.744+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:49.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.744+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:49.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.744+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:49.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:49.744+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:50.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:50.545+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:50.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:50.545+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:50.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:50.545+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:50.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:50.545+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.142+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.142+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c900 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.142+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.142+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.142+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 0x7fdfb4006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:52.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.142+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40117f0 0x7fdfb4011c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:52.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.142+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40117f0 2024-04-22T07:08:52.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.142+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb400f380 2024-04-22T07:08:52.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.142+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40117f0 msgr2=0x7fdfb4011c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:52.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.142+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40117f0 0x7fdfb4011c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:52.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.142+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:52.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.142+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.343+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40117f0 msgr2=0x7fdfb4011c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:52.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.343+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40117f0 0x7fdfb4011c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:52.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.343+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:52.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.343+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.744+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40117f0 msgr2=0x7fdfb4011c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.744+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40117f0 0x7fdfb4011c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.744+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:52.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:52.744+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:53.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:53.545+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40117f0 msgr2=0x7fdfb4011c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:53.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:53.545+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40117f0 0x7fdfb4011c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:53.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:53.545+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:53.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:53.545+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:55.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:55.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40117f0 msgr2=0x7fdfb4011c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40117f0 0x7fdfb4011c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:08:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca8d0 con 0x7fdfb400c150 2024-04-22T07:08:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400f380 2024-04-22T07:08:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb40075e0 2024-04-22T07:08:55.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:55.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:55.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:55.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:55.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.142+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.343+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:55.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.343+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:55.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.343+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:55.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.343+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:55.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.343+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:55.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.744+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:55.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.744+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:55.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.744+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:55.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.744+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:55.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:55.744+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:56.545+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:08:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:56.545+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:56.545+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:08:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:56.545+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:08:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:56.545+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:58.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.143+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:58.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.143+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:58.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.143+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:58.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.143+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:58.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.143+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:58.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.143+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012be0 0x7fdfb40063c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:58.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.143+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb40086b0 con 0x7fdfb400c8e0 2024-04-22T07:08:58.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.143+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb4012be0 2024-04-22T07:08:58.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.143+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c8e0 legacy=0x7fdfb40110e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:58.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.143+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012be0 msgr2=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:58.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.143+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012be0 0x7fdfb40063c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:58.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.344+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c8e0 legacy=0x7fdfb40110e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:58.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.344+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012be0 msgr2=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:58.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.344+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012be0 0x7fdfb40063c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.745+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c8e0 legacy=0x7fdfb40110e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.745+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012be0 msgr2=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:58.745+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012be0 0x7fdfb40063c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:59.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:59.546+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c8e0 legacy=0x7fdfb40110e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:08:59.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:59.546+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012be0 msgr2=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:08:59.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:08:59.546+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012be0 0x7fdfb40063c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:01.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012be0 msgr2=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:01.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4012be0 0x7fdfb40063c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:01.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c8e0 legacy=0x7fdfb40110e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:09:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400f380 2024-04-22T07:09:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb400c150 2024-04-22T07:09:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb4008f40 con 0x7fdfb40075e0 2024-04-22T07:09:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.143+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:01.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.344+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:01.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.344+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:01.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.344+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:01.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.344+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:01.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.344+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:01.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.344+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:01.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.745+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:01.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.745+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:01.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.745+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:01.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.745+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:01.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.745+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:01.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:01.745+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:02.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:02.546+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:02.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:02.546+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:02.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:02.546+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:02.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:02.546+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:02.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:02.546+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:02.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:02.546+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.143+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.143+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.143+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.143+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.143+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.143+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.143+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca8d0 con 0x7fdfb4012110 2024-04-22T07:09:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.143+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb40086b0 con 0x7fdfb40124f0 2024-04-22T07:09:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.143+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4012110 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.143+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb40110c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:04.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.344+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4012110 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:04.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.344+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb40110c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.745+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4012110 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:04.745+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb40110c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:05.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:05.546+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4012110 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:05.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:05.546+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb40110c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:06.150 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:09:06.153 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:09:07.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb40110c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:07.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4012110 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:07.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:09:07.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:07.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:07.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb40124f0 2024-04-22T07:09:07.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40075e0 2024-04-22T07:09:07.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb400c150 2024-04-22T07:09:07.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:07.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:07.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:07.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:07.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.144+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:07.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.345+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:07.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.345+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:07.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.345+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:07.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.345+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:07.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.345+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:07.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.746+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:07.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.746+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:07.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.746+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:07.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.746+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:07.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:07.746+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:08.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:08.547+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:08.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:08.547+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:08.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:08.547+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:08.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:08.547+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:08.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:08.547+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.143+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.143+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.143+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.143+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.143+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:10.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.143+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.143+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca8d0 con 0x7fdfb40110c0 2024-04-22T07:09:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.143+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400f380 2024-04-22T07:09:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.143+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.143+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40110c0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:10.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.143+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.344+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40110c0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.344+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.344+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:10.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.745+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40110c0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:10.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.745+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:10.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:10.745+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:11.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:11.546+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40110c0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:11.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:11.546+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:11.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:11.546+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.143+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.143+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.143+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40110c0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.144+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:09:13.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.144+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:13.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.144+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:13.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.144+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb40124f0 2024-04-22T07:09:13.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.144+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400c150 2024-04-22T07:09:13.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.144+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb40075e0 2024-04-22T07:09:13.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.144+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:13.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.144+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:13.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.144+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:13.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.144+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:13.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.144+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.345+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.345+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.345+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.345+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.345+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:13.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.746+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:13.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.746+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:13.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.746+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:13.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.746+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:13.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:13.746+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:14.547+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:14.547+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:14.547+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:14.547+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:14.547+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.144+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb40089c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.144+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb40089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.144+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.144+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.144+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.144+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.144+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca8d0 con 0x7fdfb40110c0 2024-04-22T07:09:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.144+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400f380 2024-04-22T07:09:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.144+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.144+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.144+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40110c0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:16.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.345+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:16.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.345+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:16.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.345+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40110c0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.746+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.746+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:16.746+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40110c0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:17.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:17.547+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:17.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:17.547+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:17.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:17.547+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40110c0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:19.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:19.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400f380 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:19.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40110c0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:19.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:09:19.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:19.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:19.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb400c150 2024-04-22T07:09:19.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40075e0 2024-04-22T07:09:19.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb40124f0 2024-04-22T07:09:19.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:19.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:19.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:19.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:19.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.144+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:19.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.345+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:19.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.345+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:19.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.345+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:19.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.345+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:19.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.345+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:19.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.746+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:19.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.746+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:19.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.746+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:19.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.746+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:19.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:19.746+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:20.547+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:20.547+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:20.547+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:20.547+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:20.547+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:22.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.145+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:22.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.145+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:22.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.145+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:22.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.145+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:22.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.145+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:22.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.145+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400f380 0x7fdfb4006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:22.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.145+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca8d0 con 0x7fdfb40089c0 2024-04-22T07:09:22.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.145+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400f380 2024-04-22T07:09:22.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.145+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400f380 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:22.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.145+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40089c0 legacy=0x7fdfb40107c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:22.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.145+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400f380 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.346+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400f380 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.346+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40089c0 legacy=0x7fdfb40107c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.346+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400f380 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.747+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40089c0 legacy=0x7fdfb40107c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.747+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400f380 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:22.747+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400f380 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:23.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:23.548+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40089c0 legacy=0x7fdfb40107c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:23.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:23.548+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400f380 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:23.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:23.548+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400f380 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:25.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400f380 msgr2=0x7fdfb4006cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:25.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400f380 0x7fdfb4006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:25.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40089c0 legacy=0x7fdfb40107c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:25.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:09:25.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:25.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:25.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb400c150 2024-04-22T07:09:25.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40075e0 2024-04-22T07:09:25.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb40124f0 2024-04-22T07:09:25.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfd2d9d640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:25.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:25.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfd2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:25.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:25.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.145+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:25.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.346+0000 7fdfd2d9d640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.346+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.346+0000 7fdfd2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.346+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.346+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:25.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.747+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:25.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.747+0000 7fdfd2d9d640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:25.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.747+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:25.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.747+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:25.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:25.747+0000 7fdfd2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:26.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:26.548+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:26.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:26.548+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:26.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:26.548+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:26.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:26.548+0000 7fdfd2d9d640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:26.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:26.548+0000 7fdfd2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:28.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.145+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:28.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.145+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:28.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.145+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.145+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.145+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.145+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40167b0 0x7fdfb4018ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.145+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca8d0 con 0x7fdfb4006cc0 2024-04-22T07:09:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.145+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40167b0 2024-04-22T07:09:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.145+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4006cc0 legacy=0x7fdfb40107c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.145+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40167b0 msgr2=0x7fdfb4018ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.145+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40167b0 0x7fdfb4018ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:28.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.346+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4006cc0 legacy=0x7fdfb40107c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:28.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.346+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40167b0 msgr2=0x7fdfb4018ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:28.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.346+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40167b0 0x7fdfb4018ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:28.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.747+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40167b0 msgr2=0x7fdfb4018ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:28.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.747+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4006cc0 legacy=0x7fdfb40107c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:28.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:28.747+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40167b0 0x7fdfb4018ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:29.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:29.548+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40167b0 msgr2=0x7fdfb4018ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:29.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:29.548+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40167b0 0x7fdfb4018ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:29.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:29.548+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4006cc0 legacy=0x7fdfb40107c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:31.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40167b0 msgr2=0x7fdfb4018ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:31.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40167b0 0x7fdfb4018ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:31.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4006cc0 legacy=0x7fdfb40107c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:31.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:09:31.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:31.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:31.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb40075e0 2024-04-22T07:09:31.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400c150 2024-04-22T07:09:31.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb40124f0 2024-04-22T07:09:31.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:31.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfd2d9d640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:31.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfd2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:31.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:31.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.145+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:31.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.346+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:31.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.346+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:31.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.346+0000 7fdfd2d9d640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:31.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.346+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:31.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.346+0000 7fdfd2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:31.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.747+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:31.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.747+0000 7fdfd2d9d640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:31.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.747+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:31.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.747+0000 7fdfd2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:31.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:31.747+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:32.548+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:32.548+0000 7fdfd2d9d640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:32.548+0000 7fdfd2d9d640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:32.548+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:32.548+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.146+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.146+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40124f0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.146+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.146+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.146+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.146+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb4018ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.146+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca8d0 con 0x7fdfb40107c0 2024-04-22T07:09:34.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.146+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb4010ba0 2024-04-22T07:09:34.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.146+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb4018ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:34.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.146+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb4018ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:34.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.146+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40107c0 legacy=0x7fdfb40157a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:34.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.347+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb4018ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:34.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.347+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb4018ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:34.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.347+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40107c0 legacy=0x7fdfb40157a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:34.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.748+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb4018ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:34.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.748+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb4018ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:34.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:34.748+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40107c0 legacy=0x7fdfb40157a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:35.549+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb4018ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:35.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:35.549+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb4018ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:35.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:35.549+0000 7fdfd259c640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40107c0 legacy=0x7fdfb40157a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:36.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:09:36.298 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:09:37.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb4018ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:37.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb4018ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:37.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40107c0 legacy=0x7fdfb40157a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:37.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:09:37.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb40100b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb4010ba0 2024-04-22T07:09:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40075e0 2024-04-22T07:09:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb400c150 2024-04-22T07:09:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.146+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb40100b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:37.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.347+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:37.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.347+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:37.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.347+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb40100b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:37.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.347+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:37.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.347+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:37.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.748+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:37.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.748+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:37.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.748+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:37.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.748+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb40100b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:37.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:37.748+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:38.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:38.549+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:38.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:38.549+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb40100b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:38.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:38.549+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:38.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:38.549+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:38.549+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:40.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.146+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:40.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.146+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb40100b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:40.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.146+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:40.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.146+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:40.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.146+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:40.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.146+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 0x7fdfb40197d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:40.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.146+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca8d0 con 0x7fdfb40124f0 2024-04-22T07:09:40.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.146+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400ba00 2024-04-22T07:09:40.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.146+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:40.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.146+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 msgr2=0x7fdfb40197d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:40.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.146+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 0x7fdfb40197d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:40.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.347+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:40.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.347+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 msgr2=0x7fdfb40197d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:40.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.347+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 0x7fdfb40197d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.748+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.748+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 msgr2=0x7fdfb40197d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:40.748+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 0x7fdfb40197d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:41.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:41.549+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 msgr2=0x7fdfb40197d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:41.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:41.549+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:41.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:41.549+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 0x7fdfb40197d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:43.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 msgr2=0x7fdfb40197d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:43.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400ba00 0x7fdfb40197d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:43.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40124f0 legacy=0x7fdfb40063c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:43.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:09:43.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:43.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb400c150 2024-04-22T07:09:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb4010ba0 2024-04-22T07:09:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb40075e0 2024-04-22T07:09:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.147+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:43.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.348+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:43.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.348+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:43.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.348+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:43.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.348+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:43.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.348+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.749+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.749+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.749+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.749+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:43.749+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:44.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:44.550+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:44.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:44.550+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:44.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:44.550+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:44.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:44.550+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:44.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:44.550+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:46.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.146+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:46.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.146+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.146+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.146+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.146+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.146+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4013790 0x7fdfb4015b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.146+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca8d0 con 0x7fdfb40160c0 2024-04-22T07:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.146+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb4013790 2024-04-22T07:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.146+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40160c0 legacy=0x7fdfb40184b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.146+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4013790 msgr2=0x7fdfb4015b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.146+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4013790 0x7fdfb4015b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:46.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.347+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40160c0 legacy=0x7fdfb40184b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:46.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.347+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4013790 msgr2=0x7fdfb4015b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:46.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.347+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4013790 0x7fdfb4015b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.748+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40160c0 legacy=0x7fdfb40184b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.748+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4013790 msgr2=0x7fdfb4015b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:46.748+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4013790 0x7fdfb4015b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:47.550+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40160c0 legacy=0x7fdfb40184b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:47.550+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4013790 msgr2=0x7fdfb4015b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:47.550+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4013790 0x7fdfb4015b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:49.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.146+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4013790 msgr2=0x7fdfb4015b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:49.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.146+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4013790 0x7fdfb4015b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:49.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.147+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40160c0 legacy=0x7fdfb40184b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:49.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.147+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:09:49.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.147+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:49.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.147+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb4010ba0 2024-04-22T07:09:49.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.147+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca8d0 con 0x7fdfb400c150 2024-04-22T07:09:49.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.147+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40075e0 2024-04-22T07:09:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.147+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.147+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.147+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.147+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:49.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.348+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:49.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.348+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:49.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.348+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:49.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.348+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:49.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.749+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:49.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.749+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:49.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.749+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:49.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:49.749+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:50.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:50.550+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:50.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:50.550+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:50.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:50.550+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:50.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:50.550+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:52.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.147+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.147+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.147+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.147+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4013790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.147+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400ba00 0x7fdfb4016ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.147+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4017400 0x7fdfb40197f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.147+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb4017400 2024-04-22T07:09:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.147+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb400ba00 2024-04-22T07:09:52.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.147+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4017400 msgr2=0x7fdfb40197f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:52.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.147+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4017400 0x7fdfb40197f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:52.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.147+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400ba00 msgr2=0x7fdfb4016ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:52.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.147+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400ba00 0x7fdfb4016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.348+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4017400 msgr2=0x7fdfb40197f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.348+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4017400 0x7fdfb40197f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.348+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400ba00 msgr2=0x7fdfb4016ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.348+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400ba00 0x7fdfb4016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:52.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.749+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4017400 msgr2=0x7fdfb40197f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:52.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.749+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4017400 0x7fdfb40197f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:52.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.749+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400ba00 msgr2=0x7fdfb4016ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:52.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:52.749+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400ba00 0x7fdfb4016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:53.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:53.550+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4017400 msgr2=0x7fdfb40197f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:53.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:53.550+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4017400 0x7fdfb40197f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:53.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:53.550+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400ba00 msgr2=0x7fdfb4016ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:53.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:53.550+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400ba00 0x7fdfb4016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:55.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400ba00 msgr2=0x7fdfb4016ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:55.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400ba00 0x7fdfb4016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:55.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4017400 msgr2=0x7fdfb40197f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:55.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4017400 0x7fdfb40197f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:55.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:09:55.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:55.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb40099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:55.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb40086b0 con 0x7fdfb400c150 2024-04-22T07:09:55.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb4010ba0 2024-04-22T07:09:55.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb40075e0 2024-04-22T07:09:55.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:55.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb40099f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:55.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb40099f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:55.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:55.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.147+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.348+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb40099f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.348+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.348+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb40099f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.348+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.348+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:55.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.749+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:55.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.749+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb40099f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:55.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.749+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb40099f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:55.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.749+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:55.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:55.749+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:56.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:56.550+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:09:56.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:56.550+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb40099f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:09:56.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:56.550+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb40099f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:56.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:56.550+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:09:56.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:56.550+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:58.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.148+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.148+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.148+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb40099f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.148+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb4010ba0 0x7fdfb40099f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.148+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb400c150 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.148+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4009f30 0x7fdfb400a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:58.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.148+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb400a850 2024-04-22T07:09:58.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.148+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb4009f30 2024-04-22T07:09:58.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.148+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400a850 legacy=0x7fdfb400ac60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:58.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.148+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4009f30 msgr2=0x7fdfb400a310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:58.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.148+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4009f30 0x7fdfb400a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:58.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.349+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4009f30 msgr2=0x7fdfb400a310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:58.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.349+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400a850 legacy=0x7fdfb400ac60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:58.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.349+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4009f30 0x7fdfb400a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:58.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.750+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400a850 legacy=0x7fdfb400ac60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:58.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.750+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4009f30 msgr2=0x7fdfb400a310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:58.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:58.750+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4009f30 0x7fdfb400a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:59.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:59.551+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400a850 legacy=0x7fdfb400ac60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:09:59.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:59.551+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4009f30 msgr2=0x7fdfb400a310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:09:59.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:09:59.551+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4009f30 0x7fdfb400a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:01.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4009f30 msgr2=0x7fdfb400a310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:01.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4009f30 0x7fdfb400a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:01.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb400a850 legacy=0x7fdfb400ac60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:01.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:10:01.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:01.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb40100b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:01.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb40086b0 con 0x7fdfb4010ba0 2024-04-22T07:10:01.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40075e0 2024-04-22T07:10:01.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb400c150 2024-04-22T07:10:01.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:10:01.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:01.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:10:01.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:10:01.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.148+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb40100b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:01.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.349+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:10:01.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.349+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:01.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.349+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:10:01.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.349+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:10:01.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.349+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb40100b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.750+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:10:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.750+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.750+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:10:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.750+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:10:01.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:01.750+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb40100b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:02.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:02.551+0000 7fdfd259c640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:10:02.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:02.551+0000 7fdfd259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:02.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:02.551+0000 7fdfd1d9b640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:10:02.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:02.551+0000 7fdfd1d9b640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb40100b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:02.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:02.551+0000 7fdfd2d9d640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:10:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.148+0000 7fdfc3fff640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 msgr2=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.148+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb400c150 0x7fdfb40100b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.148+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.148+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb40075e0 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.148+0000 7fdfc3fff640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb4010ba0 legacy=0x7fdfb4005cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.148+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40063c0 0x7fdfb400ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.148+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb40067a0 2024-04-22T07:10:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.148+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb40063c0 2024-04-22T07:10:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.148+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40067a0 legacy=0x7fdfb400b1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:10:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.148+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40063c0 msgr2=0x7fdfb400ac60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:10:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.148+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40063c0 0x7fdfb400ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:04.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.349+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40067a0 legacy=0x7fdfb400b1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:10:04.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.349+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40063c0 msgr2=0x7fdfb400ac60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:10:04.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.349+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40063c0 0x7fdfb400ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:04.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.750+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40067a0 legacy=0x7fdfb400b1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:10:04.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.750+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40063c0 msgr2=0x7fdfb400ac60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:10:04.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:04.750+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40063c0 0x7fdfb400ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:05.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:05.551+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40067a0 legacy=0x7fdfb400b1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:10:05.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:05.551+0000 7fdfd259c640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40063c0 msgr2=0x7fdfb400ac60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:10:05.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:05.551+0000 7fdfd259c640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40063c0 0x7fdfb400ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:06.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:10:06.427 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:10:07.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.148+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40063c0 msgr2=0x7fdfb400ac60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:07.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.148+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb40063c0 0x7fdfb400ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:07.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.148+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40067a0 legacy=0x7fdfb400b1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:07.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.148+0000 7fdfd2d9d640 1 -- reap_dead start 2024-04-22T07:10:07.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.148+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:07.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.148+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4010ba0 0x7fdfb400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:07.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.148+0000 7fdfc3fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb40086b0 con 0x7fdfb40075e0 2024-04-22T07:10:07.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.148+0000 7fdfc3fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb400c150 2024-04-22T07:10:07.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.148+0000 7fdfc3fff640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40ca5f0 con 0x7fdfb4010ba0 2024-04-22T07:10:07.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.149+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:10:07.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.149+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:10:07.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.149+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:07.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.149+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:10:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.149+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4010ba0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:07.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.350+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:10:07.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.350+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:10:07.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.350+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:07.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.350+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:10:07.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.350+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4010ba0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:07.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.751+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:10:07.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.751+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:10:07.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.751+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:07.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.751+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:10:07.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:07.751+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4010ba0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:08.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:08.552+0000 7fdfd1d9b640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-04-22T07:10:08.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:08.552+0000 7fdfd1d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:08.552+0000 7fdfd259c640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-04-22T07:10:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:08.552+0000 7fdfd2d9d640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3300/0 2024-04-22T07:10:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:08.552+0000 7fdfd2d9d640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4010ba0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.149+0000 7fdfc3fff640 1 -- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4010ba0 msgr2=0x7fdfb400ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.149+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfb4010ba0 0x7fdfb400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.149+0000 7fdfc3fff640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c150 msgr2=0x7fdfb4011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.149+0000 7fdfc3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdfb400c150 0x7fdfb4011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.149+0000 7fdfc3fff640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7fdfb40075e0 legacy=0x7fdfb40100b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.149+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4013790 0x7fdfb4015b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.149+0000 7fdfc3fff640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40ca310 con 0x7fdfb40160c0 2024-04-22T07:10:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.149+0000 7fdfc3fff640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40cb200 con 0x7fdfb4013790 2024-04-22T07:10:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.149+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4013790 msgr2=0x7fdfb4015b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:10:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.149+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40160c0 legacy=0x7fdfb40184b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:10:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.149+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4013790 0x7fdfb4015b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.350+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40160c0 legacy=0x7fdfb40184b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:10:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.350+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4013790 msgr2=0x7fdfb4015b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:10:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.350+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4013790 0x7fdfb4015b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:10.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.751+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4013790 msgr2=0x7fdfb4015b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:10:10.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.751+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40160c0 legacy=0x7fdfb40184b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:10:10.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:10.751+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4013790 0x7fdfb4015b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:11.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:11.552+0000 7fdfd1d9b640 1 -- >> v1:172.21.15.145:6789/0 conn(0x7fdfb40160c0 legacy=0x7fdfb40184b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.145:6789/0 2024-04-22T07:10:11.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:11.552+0000 7fdfd259c640 1 -- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4013790 msgr2=0x7fdfb4015b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.145:3301/0 2024-04-22T07:10:11.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-22T07:10:11.552+0000 7fdfd259c640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfb4013790 0x7fdfb4015b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:13.065 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-22T07:10:13.066 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-22T07:10:13.085 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-22T07:10:13.085 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-22T07:10:13.085 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:13.089 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-04-22T07:10:13.109 INFO:tasks.ceph.mds.a:Stopped 2024-04-22T07:10:13.109 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-22T07:10:13.109 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:13.109 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-04-22T07:10:13.125 INFO:tasks.ceph.mds.c:Stopped 2024-04-22T07:10:13.125 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-04-22T07:10:13.125 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.125 INFO:tasks.ceph.mds.e:Stopped 2024-04-22T07:10:13.126 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-22T07:10:13.126 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:13.126 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi145 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2024-04-22T07:10:13.145 INFO:tasks.ceph.mds.b:Stopped 2024-04-22T07:10:13.145 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-22T07:10:13.145 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.145 INFO:tasks.ceph.mds.d:Stopped 2024-04-22T07:10:13.145 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-04-22T07:10:13.145 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:13.146 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi145 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f' 2024-04-22T07:10:13.163 INFO:tasks.ceph.mds.f:Stopped 2024-04-22T07:10:13.163 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-22T07:10:13.163 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-22T07:10:13.163 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.163 INFO:tasks.ceph.osd.0:Stopped 2024-04-22T07:10:13.164 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-22T07:10:13.164 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.164 INFO:tasks.ceph.osd.1:Stopped 2024-04-22T07:10:13.164 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-22T07:10:13.164 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.164 INFO:tasks.ceph.osd.2:Stopped 2024-04-22T07:10:13.165 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-22T07:10:13.165 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.165 INFO:tasks.ceph.osd.3:Stopped 2024-04-22T07:10:13.165 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-22T07:10:13.165 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.165 INFO:tasks.ceph.osd.4:Stopped 2024-04-22T07:10:13.166 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-22T07:10:13.166 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.166 INFO:tasks.ceph.osd.5:Stopped 2024-04-22T07:10:13.166 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-22T07:10:13.166 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.166 INFO:tasks.ceph.osd.6:Stopped 2024-04-22T07:10:13.166 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-22T07:10:13.167 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.167 INFO:tasks.ceph.osd.7:Stopped 2024-04-22T07:10:13.167 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-22T07:10:13.167 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-22T07:10:13.167 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.167 INFO:tasks.ceph.mgr.y:Stopped 2024-04-22T07:10:13.168 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-22T07:10:13.168 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:13.168 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi145 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x' 2024-04-22T07:10:13.196 INFO:tasks.ceph.mgr.x:Stopped 2024-04-22T07:10:13.196 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-22T07:10:13.196 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-22T07:10:13.196 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.196 INFO:tasks.ceph.mon.a:Stopped 2024-04-22T07:10:13.197 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-22T07:10:13.197 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.197 INFO:tasks.ceph.mon.b:Stopped 2024-04-22T07:10:13.197 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-22T07:10:13.197 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:13.197 INFO:tasks.ceph.mon.c:Stopped 2024-04-22T07:10:13.198 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-22T07:10:13.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-04-22T07:10:13.453 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-22T07:10:13.453 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-22T07:10:13.552 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-22T07:10:13.552 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-22T07:10:13.650 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-22T07:10:13.651 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-22T07:10:13.700 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-22T07:10:13.700 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-22T07:10:13.760 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi145.front.sepia.ceph.com 2024-04-22T07:10:13.760 DEBUG:teuthology.orchestra.run.smithi145:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-22T07:10:13.848 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi145.front.sepia.ceph.com 2024-04-22T07:10:13.849 DEBUG:teuthology.orchestra.run.smithi145:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-22T07:10:13.905 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi145.front.sepia.ceph.com 2024-04-22T07:10:13.905 DEBUG:teuthology.orchestra.run.smithi145:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-22T07:10:13.964 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi145.front.sepia.ceph.com 2024-04-22T07:10:13.964 DEBUG:teuthology.orchestra.run.smithi145:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-22T07:10:14.021 INFO:tasks.ceph:Archiving mon data... 2024-04-22T07:10:14.022 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159/data/mon.a.tgz 2024-04-22T07:10:14.022 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-04-22T07:10:14.039 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.8ua2Beaw4I 2024-04-22T07:10:14.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.8ua2Beaw4I 2024-04-22T07:10:14.159 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.8ua2Beaw4I 2024-04-22T07:10:14.241 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.8ua2Beaw4I is 258KB 2024-04-22T07:10:14.336 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.8ua2Beaw4I 2024-04-22T07:10:14.350 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159/data/mon.b.tgz 2024-04-22T07:10:14.351 DEBUG:teuthology.orchestra.run.smithi145:> mktemp 2024-04-22T07:10:14.369 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/tmp.ucK8d2pwb2 2024-04-22T07:10:14.370 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ucK8d2pwb2 2024-04-22T07:10:14.539 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0666 /tmp/tmp.ucK8d2pwb2 2024-04-22T07:10:14.621 DEBUG:teuthology.orchestra.remote:smithi145:/tmp/tmp.ucK8d2pwb2 is 348KB 2024-04-22T07:10:14.717 DEBUG:teuthology.orchestra.run.smithi145:> rm -fr /tmp/tmp.ucK8d2pwb2 2024-04-22T07:10:14.732 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159/data/mon.c.tgz 2024-04-22T07:10:14.732 DEBUG:teuthology.orchestra.run.smithi145:> mktemp 2024-04-22T07:10:14.786 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/tmp.PrRgwEjjWs 2024-04-22T07:10:14.786 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.PrRgwEjjWs 2024-04-22T07:10:14.911 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0666 /tmp/tmp.PrRgwEjjWs 2024-04-22T07:10:14.992 DEBUG:teuthology.orchestra.remote:smithi145:/tmp/tmp.PrRgwEjjWs is 348KB 2024-04-22T07:10:15.086 DEBUG:teuthology.orchestra.run.smithi145:> rm -fr /tmp/tmp.PrRgwEjjWs 2024-04-22T07:10:15.101 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-22T07:10:15.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-22T07:10:15.104 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-22T07:10:15.183 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-22T07:10:15.218 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-22T07:10:15.219 INFO:tasks.ceph:Archiving crash dumps... 2024-04-22T07:10:15.220 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159/remote/smithi092/crash 2024-04-22T07:10:15.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-22T07:10:15.274 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159/remote/smithi145/crash 2024-04-22T07:10:15.275 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-22T07:10:15.319 INFO:tasks.ceph:Shutting down logrotate 2024-04-22T07:10:15.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:10:15.322 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:10:15.402 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-22T07:10:15.426 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-22T07:10:15.449 INFO:tasks.ceph:Compressing logs... 2024-04-22T07:10:15.449 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T07:10:15.469 DEBUG:teuthology.orchestra.run.smithi145:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T07:10:15.525 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-22T07:10:15.525 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33506.log 2024-04-22T07:10:15.525 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-04-22T07:10:15.525 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.33506.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-22T07:10:15.525 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mds.f.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33506.log.gz 2024-04-22T07:10:15.525 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33466.log 2024-04-22T07:10:15.526 INFO:teuthology.orchestra.run.smithi145.stderr: 91.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-22T07:10:15.526 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-22T07:10:15.526 INFO:teuthology.orchestra.run.smithi145.stderr:gzip/var/log/ceph/ceph-client.admin.33466.log: -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-22T07:10:15.526 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-22T07:10:15.526 INFO:teuthology.orchestra.run.smithi145.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33466.log.gz 2024-04-22T07:10:15.526 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33419.log 2024-04-22T07:10:15.526 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-22T07:10:15.527 INFO:teuthology.orchestra.run.smithi145.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-22T07:10:15.527 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.33419.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-22T07:10:15.527 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-osd.4.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33419.log.gz 2024-04-22T07:10:15.527 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-22T07:10:15.527 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-22T07:10:15.527 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33586.log 2024-04-22T07:10:15.527 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-osd.6.log: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-22T07:10:15.541 INFO:teuthology.orchestra.run.smithi145.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-22T07:10:15.541 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33375.log 2024-04-22T07:10:15.541 INFO:teuthology.orchestra.run.smithi145.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-04-22T07:10:15.541 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.33586.log: gzip -5 --verbose 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33586.log.gz 2024-04-22T07:10:15.541 INFO:teuthology.orchestra.run.smithi145.stderr: -- /var/log/ceph/ceph-client.admin.33335.log 2024-04-22T07:10:15.541 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.33375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33295.log 88.2% 2024-04-22T07:10:15.541 INFO:teuthology.orchestra.run.smithi145.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33375.log.gz 2024-04-22T07:10:15.541 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.33335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33546.log 2024-04-22T07:10:15.542 INFO:teuthology.orchestra.run.smithi145.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33335.log.gz 2024-04-22T07:10:15.542 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.33295.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-22T07:10:15.542 INFO:teuthology.orchestra.run.smithi145.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33295.log.gz 2024-04-22T07:10:15.542 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.33546.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-22T07:10:15.542 INFO:teuthology.orchestra.run.smithi145.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33546.log.gz 2024-04-22T07:10:15.542 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 90.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-22T07:10:15.567 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43986.log 2024-04-22T07:10:15.567 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41502.log 2024-04-22T07:10:15.567 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39451.log 2024-04-22T07:10:15.567 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44254.log 2024-04-22T07:10:15.567 INFO:teuthology.orchestra.run.smithi092.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43986.log.gz 2024-04-22T07:10:15.568 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39451.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41502.log.gz 2024-04-22T07:10:15.568 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40830.log 2024-04-22T07:10:15.568 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41366.log 2024-04-22T07:10:15.568 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39451.log.gz 2024-04-22T07:10:15.568 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41386.log 2024-04-22T07:10:15.568 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44254.log.gz 2024-04-22T07:10:15.568 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40830.log.gz 2024-04-22T07:10:15.569 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39827.log 2024-04-22T07:10:15.569 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41386.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41366.log.gz 2024-04-22T07:10:15.569 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40828.log 2024-04-22T07:10:15.569 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43692.log 87.9% 2024-04-22T07:10:15.569 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41386.log.gz 2024-04-22T07:10:15.569 INFO:teuthology.orchestra.run.smithi092.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39827.log.gz/var/log/ceph/ceph-client.admin.40828.log: 2024-04-22T07:10:15.569 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44070.log 2024-04-22T07:10:15.570 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43692.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40828.log.gz 2024-04-22T07:10:15.570 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33172.log 2024-04-22T07:10:15.570 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44112.log 2024-04-22T07:10:15.570 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43692.log.gz 2024-04-22T07:10:15.570 INFO:teuthology.orchestra.run.smithi092.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44070.log.gz 2024-04-22T07:10:15.570 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.33172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41974.log 2024-04-22T07:10:15.570 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39971.log 2024-04-22T07:10:15.571 INFO:teuthology.orchestra.run.smithi092.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.33172.log.gz 2024-04-22T07:10:15.571 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44601.log 2024-04-22T07:10:15.571 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44112.log.gz 2024-04-22T07:10:15.571 INFO:teuthology.orchestra.run.smithi092.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41974.log.gz 2024-04-22T07:10:15.571 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43902.log 2024-04-22T07:10:15.571 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44601.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39971.log.gz 2024-04-22T07:10:15.571 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33536.log 2024-04-22T07:10:15.571 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41162.log 2024-04-22T07:10:15.572 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43902.log.gz 2024-04-22T07:10:15.572 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.33536.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44601.log.gz 2024-04-22T07:10:15.572 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40139.log 2024-04-22T07:10:15.572 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41162.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33536.log.gz 2024-04-22T07:10:15.572 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42310.log 2024-04-22T07:10:15.572 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40139.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41162.log.gz 2024-04-22T07:10:15.572 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-22T07:10:15.573 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40139.log.gz 2024-04-22T07:10:15.573 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40498.log 2024-04-22T07:10:15.573 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42310.log.gz 2024-04-22T07:10:15.573 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39785.log 2024-04-22T07:10:15.573 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40013.log 2024-04-22T07:10:15.573 INFO:teuthology.orchestra.run.smithi092.stderr: 84.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-22T07:10:15.573 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39785.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40498.log.gz 2024-04-22T07:10:15.573 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43566.log 2024-04-22T07:10:15.574 INFO:teuthology.orchestra.run.smithi092.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39785.log.gz 2024-04-22T07:10:15.574 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5/var/log/ceph/ceph-client.admin.40013.log: --verbose -- /var/log/ceph/ceph-client.admin.42520.log 2024-04-22T07:10:15.574 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45275.log 2024-04-22T07:10:15.574 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40013.log.gz 2024-04-22T07:10:15.574 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42520.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.43566.log.gz 2024-04-22T07:10:15.574 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44402.log 2024-04-22T07:10:15.574 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45359.log 2024-04-22T07:10:15.574 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44402.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-22T07:10:15.575 INFO:teuthology.orchestra.run.smithi092.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45275.log.gz 2024-04-22T07:10:15.575 INFO:teuthology.orchestra.run.smithi092.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.42520.log.gz 2024-04-22T07:10:15.575 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40831.log 2024-04-22T07:10:15.575 INFO:teuthology.orchestra.run.smithi092.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44402.log.gz 2024-04-22T07:10:15.575 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45359.log.gz 2024-04-22T07:10:15.575 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40834.log 2024-04-22T07:10:15.575 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42142.log 2024-04-22T07:10:15.575 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40834.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40831.log.gz 2024-04-22T07:10:15.576 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44154.log 2024-04-22T07:10:15.576 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40834.log.gz 2024-04-22T07:10:15.576 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40232.log 2024-04-22T07:10:15.576 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44336.log 2024-04-22T07:10:15.576 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42142.log.gz 2024-04-22T07:10:15.576 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40232.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44154.log.gz 2024-04-22T07:10:15.576 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41163.log 2024-04-22T07:10:15.576 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40232.log.gz 2024-04-22T07:10:15.577 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44336.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-22T07:10:15.577 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43608.log 2024-04-22T07:10:15.577 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.audit.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41163.log.gz 2024-04-22T07:10:15.577 INFO:teuthology.orchestra.run.smithi092.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44336.log.gz 2024-04-22T07:10:15.577 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44964.log 2024-04-22T07:10:15.577 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41932.log 2024-04-22T07:10:15.577 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43608.log: /var/log/ceph/ceph-client.admin.44964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42802.log 2024-04-22T07:10:15.577 INFO:teuthology.orchestra.run.smithi092.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43608.log.gz 2024-04-22T07:10:15.578 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41932.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-22T07:10:15.578 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45860.log 2024-04-22T07:10:15.578 INFO:teuthology.orchestra.run.smithi092.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41932.log.gz 2024-04-22T07:10:15.578 INFO:teuthology.orchestra.run.smithi092.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44964.log.gz 2024-04-22T07:10:15.578 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42268.log 2024-04-22T07:10:15.578 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42802.log: /var/log/ceph/ceph-client.admin.45860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40832.log 2024-04-22T07:10:15.578 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41841.log 2024-04-22T07:10:15.578 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40832.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42268.log.gz 2024-04-22T07:10:15.579 INFO:teuthology.orchestra.run.smithi092.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42802.log.gz 2024-04-22T07:10:15.579 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43944.log 2024-04-22T07:10:15.579 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40832.log.gz 2024-04-22T07:10:15.579 INFO:teuthology.orchestra.run.smithi092.stderr:gzip/var/log/ceph/ceph-client.admin.41841.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43524.log 2024-04-22T07:10:15.579 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45217.log 2024-04-22T07:10:15.579 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41841.log.gz 2024-04-22T07:10:15.579 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43524.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43944.log.gz 2024-04-22T07:10:15.579 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33433.log 2024-04-22T07:10:15.580 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43524.log.gz 2024-04-22T07:10:15.580 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45217.log: 95.5% -- replaced with /var/log/ceph/ceph-client.admin.45860.log.gz 2024-04-22T07:10:15.580 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44468.log 2024-04-22T07:10:15.580 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45217.log.gz 2024-04-22T07:10:15.580 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.33433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42184.log 2024-04-22T07:10:15.580 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44468.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-osd.3.log 2024-04-22T07:10:15.580 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33433.log.gz 2024-04-22T07:10:15.580 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42604.log 2024-04-22T07:10:15.581 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42184.log.gz 2024-04-22T07:10:15.581 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.3.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44468.log.gz 2024-04-22T07:10:15.581 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40497.log 2024-04-22T07:10:15.581 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40494.log 2024-04-22T07:10:15.581 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45443.log 2024-04-22T07:10:15.581 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40497.log.gz 2024-04-22T07:10:15.581 INFO:teuthology.orchestra.run.smithi092.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42604.log.gz 2024-04-22T07:10:15.581 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45680.log 2024-04-22T07:10:15.581 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44667.log 2024-04-22T07:10:15.582 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40494.log.gz 2024-04-22T07:10:15.582 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45680.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.45443.log.gz 2024-04-22T07:10:15.582 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-22T07:10:15.582 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40488.log 2024-04-22T07:10:15.582 INFO:teuthology.orchestra.run.smithi092.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44667.log.gz 2024-04-22T07:10:15.582 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43326.log 2024-04-22T07:10:15.582 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39167.log 2024-04-22T07:10:15.582 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.0.43326.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40488.log.gz 2024-04-22T07:10:15.583 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-22T07:10:15.583 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39167.log: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.45680.log.gz 2024-04-22T07:10:15.583 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-22T07:10:15.583 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39167.log.gz 2024-04-22T07:10:15.583 INFO:teuthology.orchestra.run.smithi092.stderr:gzip/var/log/ceph/ceph-osd.1.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44710.log 2024-04-22T07:10:15.583 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42058.log 2024-04-22T07:10:15.583 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42016.log 2024-04-22T07:10:15.583 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42058.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44710.log.gz 2024-04-22T07:10:15.583 INFO:teuthology.orchestra.run.smithi092.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.0.43326.log.gz 2024-04-22T07:10:15.584 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42058.log.gz 2024-04-22T07:10:15.584 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40492.log 2024-04-22T07:10:15.584 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42016.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-22T07:10:15.584 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40492.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42016.log.gz 2024-04-22T07:10:15.584 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41030.log 2024-04-22T07:10:15.584 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40492.log.gz 2024-04-22T07:10:15.584 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44837.log 2024-04-22T07:10:15.584 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.41030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39691.log 2024-04-22T07:10:15.584 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0%/var/log/ceph/ceph-client.admin.44837.log: -- replaced with /var/log/ceph/ceph-client.admin.41030.log.gz 2024-04-22T07:10:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44535.log 2024-04-22T07:10:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39691.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44837.log.gz 2024-04-22T07:10:15.585 INFO:teuthology.orchestra.run.smithi092.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39691.log.gz 2024-04-22T07:10:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40821.log 2024-04-22T07:10:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42226.log 2024-04-22T07:10:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40821.log: 91.3% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40821.log.gz 2024-04-22T07:10:15.585 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44535.log.gz 2024-04-22T07:10:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42478.log 2024-04-22T07:10:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42670.log 2024-04-22T07:10:15.586 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42226.log.gz/var/log/ceph/ceph-client.admin.42478.log: 2024-04-22T07:10:15.586 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42736.log 2024-04-22T07:10:15.586 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42478.log.gz 2024-04-22T07:10:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44794.log 2024-04-22T07:10:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42736.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42670.log.gz 2024-04-22T07:10:15.586 INFO:teuthology.orchestra.run.smithi092.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42736.log.gz 2024-04-22T07:10:15.586 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41168.log 2024-04-22T07:10:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41503.log 2024-04-22T07:10:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41168.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44794.log.gz 2024-04-22T07:10:15.587 INFO:teuthology.orchestra.run.smithi092.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41168.log.gz 2024-04-22T07:10:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-22T07:10:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42394.log 2024-04-22T07:10:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mon.a.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41503.log.gz 2024-04-22T07:10:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41799.log 2024-04-22T07:10:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42394.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42394.log.gz 2024-04-22T07:10:15.587 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40823.log 2024-04-22T07:10:15.588 INFO:teuthology.orchestra.run.smithi092.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz/var/log/ceph/ceph-client.admin.41799.log: 2024-04-22T07:10:15.588 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41799.log.gz 2024-04-22T07:10:15.588 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41516.log 2024-04-22T07:10:15.588 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43734.log 2024-04-22T07:10:15.588 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41516.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40823.log.gz 2024-04-22T07:10:15.588 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39647.log 2024-04-22T07:10:15.588 INFO:teuthology.orchestra.run.smithi092.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41516.log.gz 2024-04-22T07:10:15.588 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41757.log 2024-04-22T07:10:15.588 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9%/var/log/ceph/ceph-client.admin.39647.log: -- replaced with /var/log/ceph/ceph-client.admin.43734.log.gz 2024-04-22T07:10:15.588 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39647.log.gz 2024-04-22T07:10:15.589 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41169.log 2024-04-22T07:10:15.589 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39563.log 2024-04-22T07:10:15.589 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9%/var/log/ceph/ceph-client.admin.41169.log: -- replaced with /var/log/ceph/ceph-client.admin.41757.log.gz 2024-04-22T07:10:15.589 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41169.log.gz 2024-04-22T07:10:15.589 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41166.log 2024-04-22T07:10:15.589 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41501.log 2024-04-22T07:10:15.589 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39563.log.gz 2024-04-22T07:10:15.589 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41166.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41166.log.gz 2024-04-22T07:10:15.589 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45133.log 2024-04-22T07:10:15.590 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-04-22T07:10:15.590 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41501.log: /var/log/ceph/ceph-client.admin.45133.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41501.log.gz 2024-04-22T07:10:15.590 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45133.log.gz 2024-04-22T07:10:15.590 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39382.log 2024-04-22T07:10:15.590 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44028.log 2024-04-22T07:10:15.590 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39382.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39382.log.gz 2024-04-22T07:10:15.590 INFO:teuthology.orchestra.run.smithi092.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-22T07:10:15.591 INFO:teuthology.orchestra.run.smithi092.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-04-22T07:10:15.591 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43650.log 2024-04-22T07:10:15.591 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40055.log 2024-04-22T07:10:15.591 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44028.log: /var/log/ceph/ceph-client.admin.43650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44196.log 2024-04-22T07:10:15.591 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40055.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44028.log.gz 2024-04-22T07:10:15.591 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43650.log.gz 2024-04-22T07:10:15.591 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39513.log 2024-04-22T07:10:15.592 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40055.log.gz 2024-04-22T07:10:15.592 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40190.log 2024-04-22T07:10:15.592 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44196.log: /var/log/ceph/ceph-client.admin.39513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40489.log 2024-04-22T07:10:15.592 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40190.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44196.log.gz 2024-04-22T07:10:15.592 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39513.log.gz 2024-04-22T07:10:15.592 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27142.log 2024-04-22T07:10:15.592 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40190.log.gz 2024-04-22T07:10:15.592 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45091.log 2024-04-22T07:10:15.592 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40489.log: /var/log/ceph/ceph.tmp-client.admin.27142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40486.log 2024-04-22T07:10:15.593 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27142.log.gz 2024-04-22T07:10:15.593 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45091.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40489.log.gz 2024-04-22T07:10:15.593 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45091.log.gz 2024-04-22T07:10:15.593 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43860.log 2024-04-22T07:10:15.593 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45175.log 2024-04-22T07:10:15.593 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40486.log: /var/log/ceph/ceph-client.admin.43860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41715.log 2024-04-22T07:10:15.593 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45175.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40486.log.gz 2024-04-22T07:10:15.593 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43860.log.gz 2024-04-22T07:10:15.594 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45401.log 2024-04-22T07:10:15.594 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45175.log.gz 2024-04-22T07:10:15.594 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42436.log 2024-04-22T07:10:15.594 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41715.log: /var/log/ceph/ceph-client.admin.45401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40833.log 2024-04-22T07:10:15.594 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42436.log: 87.7% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41715.log.gz 2024-04-22T07:10:15.594 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45401.log.gz 2024-04-22T07:10:15.594 INFO:teuthology.orchestra.run.smithi092.stderr:gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.42436.log.gz --verbose 2024-04-22T07:10:15.594 INFO:teuthology.orchestra.run.smithi092.stderr: -- /var/log/ceph/ceph-client.admin.40097.log 2024-04-22T07:10:15.595 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42904.log 2024-04-22T07:10:15.595 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40833.log: /var/log/ceph/ceph-client.admin.40097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43818.log 2024-04-22T07:10:15.595 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42904.log: 89.3% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40833.log.gz 2024-04-22T07:10:15.595 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40097.log.gz 2024-04-22T07:10:15.595 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42904.log.gz 2024-04-22T07:10:15.595 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-22T07:10:15.595 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42100.log 2024-04-22T07:10:15.595 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43818.log: /var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41887.log 2024-04-22T07:10:15.595 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42100.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43818.log.gz 2024-04-22T07:10:15.596 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42352.log 2024-04-22T07:10:15.596 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42100.log.gz 2024-04-22T07:10:15.596 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41887.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41887.log.gz 2024-04-22T07:10:15.596 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40274.log 2024-04-22T07:10:15.596 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41170.log 2024-04-22T07:10:15.596 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42352.log: /var/log/ceph/ceph-client.admin.40274.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42352.log.gz 2024-04-22T07:10:15.596 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39911.log 2024-04-22T07:10:15.596 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40274.log.gz 2024-04-22T07:10:15.597 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41170.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41170.log.gz 2024-04-22T07:10:15.597 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39605.log 2024-04-22T07:10:15.597 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43776.log 2024-04-22T07:10:15.597 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39911.log: /var/log/ceph/ceph-client.admin.39605.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39911.log.gz 2024-04-22T07:10:15.597 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39605.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41167.log 2024-04-22T07:10:15.597 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-22T07:10:15.597 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43776.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43776.log.gz 2024-04-22T07:10:15.597 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39282.log 2024-04-22T07:10:15.598 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43482.log 2024-04-22T07:10:15.598 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41167.log: /var/log/ceph/ceph-client.admin.39282.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41167.log.gz 2024-04-22T07:10:15.598 INFO:teuthology.orchestra.run.smithi092.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39282.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39735.log 2024-04-22T07:10:15.598 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-22T07:10:15.598 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43482.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43482.log.gz 2024-04-22T07:10:15.598 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45317.log 2024-04-22T07:10:15.598 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45049.log 2024-04-22T07:10:15.599 INFO:teuthology.orchestra.run.smithi145.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-22T07:10:15.600 INFO:teuthology.orchestra.run.smithi145.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-22T07:10:15.600 INFO:teuthology.orchestra.run.smithi145.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-22T07:10:15.603 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39735.log: /var/log/ceph/ceph-client.admin.45317.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39735.log.gz 2024-04-22T07:10:15.603 INFO:teuthology.orchestra.run.smithi092.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45317.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40316.log 2024-04-22T07:10:15.603 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-22T07:10:15.603 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45049.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45049.log.gz 2024-04-22T07:10:15.603 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39869.log 2024-04-22T07:10:15.604 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44921.log 2024-04-22T07:10:15.604 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40316.log: /var/log/ceph/ceph-client.admin.39869.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40316.log.gz 2024-04-22T07:10:15.604 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40496.log 2024-04-22T07:10:15.604 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39869.log.gz 2024-04-22T07:10:15.604 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44921.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44921.log.gz 2024-04-22T07:10:15.604 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41504.log 2024-04-22T07:10:15.604 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41508.log 2024-04-22T07:10:15.604 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40496.log: /var/log/ceph/ceph-client.admin.41504.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40496.log.gz 2024-04-22T07:10:15.604 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41504.log.gz 2024-04-22T07:10:15.605 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41508.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41508.log.gz 2024-04-22T07:10:15.605 INFO:teuthology.orchestra.run.smithi092.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-22T07:10:15.605 INFO:teuthology.orchestra.run.smithi092.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-22T07:10:15.605 INFO:teuthology.orchestra.run.smithi092.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-22T07:10:15.606 INFO:teuthology.orchestra.run.smithi092.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-22T07:10:15.606 INFO:teuthology.orchestra.run.smithi145.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-22T07:10:15.618 INFO:teuthology.orchestra.run.smithi092.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-22T07:10:15.618 INFO:teuthology.orchestra.run.smithi145.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-22T07:10:15.618 INFO:teuthology.orchestra.run.smithi145.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-22T07:10:15.620 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-04-22T07:10:15.620 INFO:teuthology.orchestra.run.smithi145.stderr:real 0m0.117s 2024-04-22T07:10:15.620 INFO:teuthology.orchestra.run.smithi145.stderr:user 0m0.529s 2024-04-22T07:10:15.621 INFO:teuthology.orchestra.run.smithi145.stderr:sys 0m0.043s 2024-04-22T07:10:15.655 INFO:teuthology.orchestra.run.smithi092.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-22T07:10:15.657 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-22T07:10:15.657 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m0.176s 2024-04-22T07:10:15.657 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m0.633s 2024-04-22T07:10:15.657 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m0.105s 2024-04-22T07:10:15.658 INFO:tasks.ceph:Archiving logs... 2024-04-22T07:10:15.658 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159/remote/smithi092/log 2024-04-22T07:10:15.659 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T07:10:15.912 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159/remote/smithi145/log 2024-04-22T07:10:15.913 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T07:10:16.016 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-22T07:10:16.034 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-22T07:10:16.057 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2024-04-22T07:10:16.072 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-22T07:10:16.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T07:10:16.075 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T07:10:16.164 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-22T07:10:16.174 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-22T07:10:16.175 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:10:16.178 DEBUG:teuthology.orchestra.run.smithi145:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:10:16.189 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:10:16.191 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:10:16.193 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:10:16.193 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-04-22T07:10:16.194 INFO:teuthology.orchestra.run.smithi092.stdout:^- hv01.front.sepia.ceph.com 2 6 377 52 +1210us[+1211us] +/- 590ms 2024-04-22T07:10:16.194 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv02.front.sepia.ceph.com 3 6 377 51 -51us[ -51us] +/- 62ms 2024-04-22T07:10:16.194 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +56us[ +56us] +/- 66ms 2024-04-22T07:10:16.194 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:10:16.195 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:10:16.195 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-04-22T07:10:16.195 INFO:teuthology.orchestra.run.smithi145.stdout:^- hv01.front.sepia.ceph.com 2 6 377 53 +1211us[+1246us] +/- 590ms 2024-04-22T07:10:16.195 INFO:teuthology.orchestra.run.smithi145.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -44us[-9209ns] +/- 62ms 2024-04-22T07:10:16.195 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +59us[ +93us] +/- 66ms 2024-04-22T07:10:16.196 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:10:16.196 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-22T07:10:16.257 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-22T07:10:16.257 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-22T07:10:16.267 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T07:10:16.328 DEBUG:teuthology.orchestra.run.smithi145:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T07:10:16.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T07:10:16.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:16.410 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T07:10:16.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:16.436 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-22T07:10:16.445 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-22T07:10:16.454 INFO:teuthology.task.internal:Duration was 784.513646 seconds 2024-04-22T07:10:16.455 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-22T07:10:16.463 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-22T07:10:16.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T07:10:16.466 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T07:10:16.499 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T07:10:16.512 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T07:10:16.891 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-22T07:10:16.892 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-04-22T07:10:16.892 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T07:10:16.951 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi145.front.sepia.ceph.com 2024-04-22T07:10:16.951 DEBUG:teuthology.orchestra.run.smithi145:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T07:10:16.974 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-22T07:10:16.974 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T07:10:16.993 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T07:10:17.039 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-22T07:10:17.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T07:10:17.057 DEBUG:teuthology.orchestra.run.smithi145:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T07:10:17.173 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-22T07:10:17.183 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-22T07:10:17.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T07:10:17.207 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T07:10:17.230 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-22T07:10:17.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:10:17.249 DEBUG:teuthology.orchestra.run.smithi145:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:10:17.268 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-04-22T07:10:17.292 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = core 2024-04-22T07:10:17.314 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:10:17.339 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi092.front.sepia.ceph.com, flagging run as failed 2024-04-22T07:10:17.339 DEBUG:teuthology.orchestra.run.smithi145:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:10:17.368 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi145.front.sepia.ceph.com, flagging run as failed 2024-04-22T07:10:17.368 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-22T07:10:17.379 INFO:teuthology.task.internal:Transferring archived files... 2024-04-22T07:10:17.379 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159/remote/smithi092 2024-04-22T07:10:17.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T07:10:34.897 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2024-04-22T07:10:34.910 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159/remote/smithi092/coredump/1713769521.39284.core.gz: gzip compressed data, was "1713769521.39284.core", last modified: Mon Apr 22 07:10:26 2024, max compression, original size modulo 2^32 265445376 2024-04-22T07:10:34.911 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2024-04-22T07:10:36.027 INFO:teuthology.task.internal: core looks like: /tmp/tmpohofh1kj: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-04-22T07:10:36.054 INFO:teuthology.task.internal: dump_program: ceph-mds -f --cluster ceph -i c 2024-04-22T07:10:36.055 DEBUG:teuthology.orchestra.run.smithi092:> which 'ceph-mds -f --cluster ceph -i c' 2024-04-22T07:10:36.092 INFO:teuthology.orchestra.run.smithi092.stderr:/usr/bin/which: invalid option -- 'f' 2024-04-22T07:10:36.093 INFO:teuthology.orchestra.run.smithi092.stderr:/usr/bin/which: unrecognized option '--cluster' 2024-04-22T07:10:36.093 INFO:teuthology.orchestra.run.smithi092.stderr:/usr/bin/which: no c in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-04-22T07:10:36.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:36.094 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-mds 2024-04-22T07:10:36.094 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph 2024-04-22T07:10:36.095 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 1: "which 'ceph-mds -f --cluster ceph -i c'" 2024-04-22T07:10:36.116 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-22T07:10:36.127 INFO:teuthology.task.internal:Not uploading archives. 2024-04-22T07:10:36.128 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-22T07:10:36.149 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-22T07:10:36.150 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T07:10:36.157 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T07:10:36.171 INFO:teuthology.orchestra.run.smithi092.stdout: 262213 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest 2024-04-22T07:10:36.171 INFO:teuthology.orchestra.run.smithi092.stdout: 410661 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 22 07:06 /home/ubuntu/cephtest/mnt.0 2024-04-22T07:10:36.172 INFO:teuthology.orchestra.run.smithi092.stdout: 410662 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 22 07:06 /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T07:10:36.172 INFO:teuthology.orchestra.run.smithi092.stdout: 410663 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 22 07:06 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T07:10:36.172 INFO:teuthology.orchestra.run.smithi092.stdout: 410665 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:06 /home/ubuntu/cephtest/mnt.0/client.0/tmp/foo 2024-04-22T07:10:36.172 INFO:teuthology.orchestra.run.smithi092.stdout: 410666 64716 -rw-r--r-- 1 ubuntu ubuntu 66266488 Apr 22 07:06 /home/ubuntu/cephtest/mnt.0/client.0/tmp/foo/linux-2.6.33.tar.bz2 2024-04-22T07:10:36.172 INFO:teuthology.orchestra.run.smithi092.stdout: 410664 64716 -rw-r--r-- 1 ubuntu ubuntu 66266488 Feb 24 2010 /home/ubuntu/cephtest/mnt.0/client.0/tmp/linux-2.6.33.tar.bz2 2024-04-22T07:10:36.172 INFO:teuthology.orchestra.run.smithi092.stdout: 262214 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest/archive 2024-04-22T07:10:36.172 INFO:teuthology.orchestra.run.smithi092.stdout: 395989 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest/archive/audit 2024-04-22T07:10:36.173 INFO:teuthology.orchestra.run.smithi092.stdout: 396272 128 -rw------- 1 ubuntu ubuntu 128226 Apr 22 07:10 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-04-22T07:10:36.173 INFO:teuthology.orchestra.run.smithi092.stdout: 262216 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest/archive/syslog 2024-04-22T07:10:36.173 INFO:teuthology.orchestra.run.smithi092.stdout: 262259 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 22 06:57 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-04-22T07:10:36.173 INFO:teuthology.orchestra.run.smithi092.stdout: 262217 184 -rw-r--r-- 1 ubuntu ubuntu 186637 Apr 22 07:10 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T07:10:36.173 INFO:teuthology.orchestra.run.smithi092.stdout: 262218 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 22 06:57 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-04-22T07:10:36.173 INFO:teuthology.orchestra.run.smithi092.stdout: 262215 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:05 /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:10:36.173 INFO:teuthology.orchestra.run.smithi092.stdout: 262389 31684 -rw------- 1 root root 262017024 Apr 22 07:05 /home/ubuntu/cephtest/archive/coredump/1713769521.39283.core 2024-04-22T07:10:36.174 INFO:teuthology.orchestra.run.smithi092.stdout: 262388 28428 -rw------- 1 root root 265445376 Apr 22 07:05 /home/ubuntu/cephtest/archive/coredump/1713769521.39284.core 2024-04-22T07:10:36.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:36.175 INFO:teuthology.orchestra.run.smithi092.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-22T07:10:36.175 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-22T07:10:36.190 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-22T07:10:36.205 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-22T07:10:36.218 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2024-04-22T07:10:36.218 INFO:teuthology.nuke:Checking targets against current locks 2024-04-22T07:10:36.234 INFO:teuthology.orchestra.run.smithi145.stdout: 262213 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest 2024-04-22T07:10:36.235 INFO:teuthology.orchestra.run.smithi145.stdout: 262214 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest/archive 2024-04-22T07:10:36.235 INFO:teuthology.orchestra.run.smithi145.stdout: 262259 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest/archive/audit 2024-04-22T07:10:36.235 INFO:teuthology.orchestra.run.smithi145.stdout: 262268 112 -rw------- 1 ubuntu ubuntu 113754 Apr 22 07:10 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-04-22T07:10:36.235 INFO:teuthology.orchestra.run.smithi145.stdout: 262216 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest/archive/syslog 2024-04-22T07:10:36.235 INFO:teuthology.orchestra.run.smithi145.stdout: 262267 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 22 06:57 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-04-22T07:10:36.235 INFO:teuthology.orchestra.run.smithi145.stdout: 262217 180 -rw-r--r-- 1 ubuntu ubuntu 182333 Apr 22 07:10 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T07:10:36.235 INFO:teuthology.orchestra.run.smithi145.stdout: 262218 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 22 06:57 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-04-22T07:10:36.236 INFO:teuthology.orchestra.run.smithi145.stdout: 262215 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:05 /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:10:36.236 INFO:teuthology.orchestra.run.smithi145.stdout: 262287 27756 -rw------- 1 root root 261701632 Apr 22 07:05 /home/ubuntu/cephtest/archive/coredump/1713769521.39379.core 2024-04-22T07:10:36.236 INFO:teuthology.orchestra.run.smithi145.stdout: 262286 27972 -rw------- 1 root root 261308416 Apr 22 07:05 /home/ubuntu/cephtest/archive/coredump/1713769521.39377.core 2024-04-22T07:10:36.236 INFO:teuthology.orchestra.run.smithi145.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-22T07:10:36.261 DEBUG:teuthology.nuke:shortname: smithi092 2024-04-22T07:10:36.262 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T07:10:36.304 DEBUG:teuthology.nuke:shortname: smithi145 2024-04-22T07:10:36.304 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T07:10:36.315 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:47.198794', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJVl31wtivxBEbjpTOCjOi5KvLj1vB5LcpWY56N+WMhEvOx32lnLtLoh1Bq8+LgeR9qUCXErSnly6rZdEGibL6Y='} 2024-04-22T07:10:36.357 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668159', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:47.197371', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDTO262E0ybwmBW+GUof6kOF2K0vgwR1F/9MxdtBjNRvP52H+6lALEgMCoJarHGCC43MXkctFfZm4ttjS+xK7XQ='} 2024-04-22T07:10:36.405 INFO:teuthology.orchestra.console.smithi092:Power off 2024-04-22T07:10:36.405 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T07:10:36.444 INFO:teuthology.orchestra.console.smithi145:Power off 2024-04-22T07:10:36.445 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T07:10:36.455 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2024-04-22T07:10:36.456 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:36.478 DEBUG:teuthology.orchestra.console.smithi145:power off output: Chassis Power Control: Down/Off 2024-04-22T07:10:36.478 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:36.505 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-04-22T07:10:36.510 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-04-22T07:10:40.506 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:40.515 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:40.617 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-04-22T07:10:40.625 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-04-22T07:10:44.623 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:44.631 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:44.733 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-04-22T07:10:44.741 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is off 2024-04-22T07:10:44.841 INFO:teuthology.orchestra.console.smithi145:Power off completed 2024-04-22T07:10:48.738 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:48.849 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2024-04-22T07:10:48.950 INFO:teuthology.orchestra.console.smithi092:Power off completed 2024-04-22T07:10:49.060 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/fs/snaps}} duration: 784.5136456489563 failure_reason: 'Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi092 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86f7587a5a09af35f0895e1a2d08527638fae697 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=af9933ec336b4d789d31a3010eb0362d status: fail success: false 2024-04-22T07:10:49.060 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T07:10:49.279 INFO:teuthology.run:FAIL